Imaging Device and Manufacturing Method Thereof

ABSTRACT

A small, low-profile imaging device that obtains imaging signals having similar light intensity distributions for different colored light, even when there is variability in component precision or assembly. The imaging device ( 101 ) includes a plurality of lens units ( 113 ) each including at least one lens, a plurality of imaging areas corresponding one-to-one with the plurality of lens units, and each having a light receiving surface ( 123 ) substantially perpendicular to an optical axis direction of the corresponding lens unit, an imaging signal input unit ( 133 ) that receives as input a plurality of imaging signals each output from a different one of the plurality of imaging areas, and an intensity correcting unit ( 142 ) that corrects the intensity of each of the plurality of imaging signals, so that the degree of correction changes depending on the position of the imaging area.

TECHNICAL FIELD

The present invention relates to a small, low-profile imaging device anda manufacturing method thereof.

BACKGROUND ART

A conventional imaging device is disclosed, for example, in JP2001-78213A. FIG. 50 is a cross-sectional view showing the configurationof the conventional imaging device.

In FIG. 50, an imaging system 9010 is an optical processing system thatimages light from an object on the imaging surface of an imaging element9120 via an aperture 9110 and an imaging lens 9100. The aperture 9110has three circular openings 9110 a, 9110 b and 9110 c. The object lightfrom the openings 9110 a, 9110 b and 9110 c that is incident on a lightincidence surface 9100 e of the imaging lens 9100 is emitted from threelens units 9100 a, 9100 b and 9100 c of the imaging lens 9110 and formsthree object images on the imaging surface of the imaging element 9120.A shading film is formed on a flat portion 9100 d of the imaging lens9100. Three optical filters 9052 a, 9052 b and 9052 c that transmitlight of different wavelength ranges are formed on the light incidencesurface 9100 e of the imaging lens 9100. Three optical filters 9053 a,9053 b and 9053 c that transmit light of different wavelength ranges arealso formed on three imaging areas 9120 a, 9120 b and 9120 c on theimaging element 9120. The optical filters 9052 a and 9053 a have aspectral transmittance characteristic of mainly transmitting green(marked G), the optical filters 9052 b and 9053 b have a spectraltransmittance characteristic of mainly transmitting red (marked R), andthe optical filters 9052 c and 9053 c have a spectral transmittancecharacteristic of mainly transmitting blue (marked B). Thus, the imagingareas 9120 a, 9120 b and 9120 c are respectively sensitive to green (G),red (R) and blue (B) light.

With an imaging device such as this having a plurality of imaginglenses, the mutual spacing between the plurality of object imagesrespectively formed by the plurality of imaging lenses on the imagingsurface of the imaging element 9120 changes when the distance from thecamera module to the object changes.

With the above conventional camera module, the optical axis spacing ofthe plurality of imaging systems is set such that the mutual spacingbetween the plurality of object images when the object is at a virtualsubject distance D[m] and the mutual spacing between the plurality ofobject images when the object is at infinity changes by less than twicethe pixel pitch of a reference image signal, where D=1.4/(tan θ/2), withthe virtual subject distance D[m] as a function of the angle of viewθ[°] of the plurality of imaging systems. That is, color shift in theimages of an object at infinity can be suppressed to a permissiblelevel, even when the same image processing optimized for capturing animage of an object at a virtual subject distance D[m] is performed on anobject at infinity, because the optical axis spacing is set such thatthe difference in mutual spacing between the two sets of object imageson the imaging surface will be less than twice the pixel pitch of areference signal.

In the conventional imaging device, the optical axes of the three lensunits 9100 a, 9100 b and 9100 c of the imaging lens 9100 are disposed soas to pass respectively through the centers of the three circularopenings 9110 a, 9110 b and 9110 c of the aperture 9110 and the centersof the imaging areas 9120 a, 9120 b and 9120 c. However, the opticalaxes of the three lens units 9100 a, 9100 b and 9100 c of the imaginglens 9100 can deviate from the respective centers of the three circularopenings 9110 a, 9110 b and 9110 c of the aperture 9110 due tovariability in component precision, assembly or the like. Acharacteristic particular to the lens is that the light intensity aroundthe periphery of the imaging surface of the imaging element 9120(peripheral brightness) decreases in comparison to the center, althoughthe extent to which peripheral brightness decreases differs when theoptical axes of the three lens units 9100 a, 9100 b and 9100 c of theimaging lens 9100 deviate in different directions from the centers ofthe three circular openings 9110 a, 9110 b and 9110 c of the aperture9110.

FIG. 51 illustrates the relationship between the aperture, the lensunits, and peripheral brightness. In FIG. 51, only the lens 9100, theaperture 9110 and the imaging element 9120 are shown for simplicity. Thecurves marked G, R and B show the respective light intensities of thecolors green, red and blue. Here, the positive sense of the y directionis upwards on the page, as shown in FIG. 51. As in FIG. 51, theperipheral brightness on the imaging surface of the imaging element 9120decreases symmetrically in the positive and negative senses of the ydirection, when the center of the circular opening 9110 b coincides withthe optical axis of the lens unit 9110 b (curved distribution marked R).Thus, the light intensity distribution for red is positively andnegatively symmetrical in relation to the y direction. However, when thecenter of the circular opening 9110 a deviates from the optical axis ofthe lens unit 9100 a in the negative sense of the y direction, theperipheral brightness on the imaging surface of the imaging element 9120decreases to a greater extent in the negative sense of the y direction(curved distribution marked G). Thus, the light intensity distributionfor green is pronounced in the positive sense in relation to the ydirection. On the other hand, when the center of the circular opening9110 c deviates from the optical axis of the lens unit 9100 c in thepositive sense of the y direction, due to variability in processingprecision of the lends 9100 or the apertures 9110, brightness on theimaging surface of the imaging element 9120 decreases to a greaterextent in the positive sense of the y direction (curved distributionmarked B). Thus, the light intensity distribution for blue is pronouncedin the negative sense in relation to the y direction. Note that when theaperture 9110 and the lens 9100 are made from thermoformed resin,variability as in FIG. 51 can arise from differences in the coefficientof thermal expansion resulting from compositional differences.

FIG. 52 shows the light intensity distributions for the green, red andblue components. The y-axis is shown on the horizontal axis and lightintensity is shown on the vertical axis. Where, for example, images of agray subject are captured and synthesized when the above variability ispresent, colors (false color) other than the actual colors (gray in thepresent example) of the subject are produced, such as red in centralportions in the y direction, green in positive positions, and blue innegative positions, since the light intensity distribution for red(curve marked R) will be positively and negatively symmetrical inrelation to the y direction, the light intensity distribution for green(curve marked G) will be pronounced in the positive sense in relation tothe y direction, and the light intensity distribution for blue (curvemarked B) will be pronounced in the negative sense in relation to the ydirection, as shown in FIG. 52. That is, a conventional imaging devicethat has a plurality of lens units and receives red, green and bluelight of the subject independently in imaging areas correspondingrespectively to the lens units produces false colors when the lightintensity distribution is biased because of differing light intensitiesfor red, green and blue light.

False colors thus are produced when the light axes of the three lensunits 9100 a, 9100 b and 9100 c of the imaging lens 9100 deviate fromthe respective centers of the circular openings 9110 a, 9110 b and 9110c of the aperture 9110 due to variability in component precision,assembly or the like.

Note that the above problem does not arise with an imaging deviceconstituted by a single lens unit and a single imaging area, and havingan imaging element in which a Bayer array of color filters is disposedin the imaging area (e.g., imaging element is a CCD, and has 3 differentcolor filters red, green and blue disposed in a lattice on the surfaceof the imaging element, each color filter corresponding to a differentphotodiode). That is, false colors are not produced even if the lensunit deviates from the center of the aperture and light intensities arebiased due to the aforementioned variability in component precision,assembly or the like, because the light intensity distributions for red,green and blue will be similar, since the red, green and blue colorfilters are disposed in a lattice in proximity to each other, and thered, green and blue light of the subject is received at an imaging areathat brings them close together. However, the size and profile of animaging device constituted by a single lens unit and a single imagingelement cannot be reduced because of the long optical length.

DISCLOSURE OF INVENTION

The present invention, which was made in consideration of the aboveproblems, has as its object to provide a small, low-profile imagingdevice that obtains imaging signals having similar light intensitydistributions for different colored light, even when there isvariability in component precision or assembly.

An imaging device of the present invention includes a plurality of lensunits each including at least one lens, a plurality of imaging areascorresponding one-to-one with the plurality of lens units, and eachhaving a light receiving surface substantially perpendicular to anoptical axis direction of the corresponding lens unit, an imaging signalinput unit that receives as input a plurality of imaging signals eachoutput from a different one of the plurality of imaging areas, anintensity correction coefficient saving unit that saves an intensitycorrection coefficient, which is information concerning intensityunevenness in the imaging areas, and an intensity correcting unit thatcorrects the intensity of each of the plurality of imaging signals usingthe intensity correction coefficient, so as to reduce the effect ofintensity unevenness in the imaging areas.

The present invention is able to provide a small, low-profile imagingdevice that obtains imaging signals having similar light intensitydistributions for different colored light, even when there isvariability in component precision or assembly.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a cross-sectional view showing the configuration of an imagingdevice according to Embodiment 1 of the present invention.

FIG. 2 is a top view of the lens of the imaging device according toEmbodiment 1 of the present invention.

FIG. 3 is a top view of the circuit unit of the imaging device accordingto Embodiment 1 of the present invention.

FIG. 4 is a characteristics diagram of color filters of the imagingdevice according to Embodiment 1 of the present invention.

FIG. 5 is a characteristics diagram of an IR filter of the imagingdevice according to Embodiment 1 of the present invention.

FIG. 6 illustrates the position of images of an object at infinity inthe imaging device according to Embodiment 1 of the present invention.

FIG. 7 illustrates the position of images of an object at a finitedistance in the imaging device according to Embodiment 1 of the presentinvention.

FIG. 8A illustrates the relationship between an in-focus image and acontrast evaluation value in the imaging device according to Embodiment1 of the present invention, and FIG. 8B illustrates the relationshipbetween an out-of-focus image and a contrast evaluation value in theimaging device according to Embodiment 1 of the present invention.

FIG. 9 illustrates the relationship between lens position and thecontrast evaluation value in the imaging device according to Embodiment1 of the present invention.

FIG. 10 is a block diagram of the imaging device according to Embodiment1 of the present invention.

FIG. 11 is a flowchart showing the operations of the imaging deviceaccording to Embodiment 1 of the present invention.

FIG. 12 is a flowchart showing an autofocus control operation accordingto Embodiment 1 of the present invention.

FIG. 13 illustrates the coordinates of an imaging signal of the imagingdevice according to Embodiment 1 of the present invention.

FIG. 14 is a flowchart showing an intensity correction operationaccording to Embodiment 1 of the present invention.

FIG. 15 illustrates distortion correction coefficients according toEmbodiment 1 of the present invention.

FIG. 16 is a flowchart showing a distortion correction operationaccording to Embodiment 1 of the present invention.

FIG. 17 is a flowchart showing a parallax correction operation accordingto Embodiment 1 of the present invention.

FIG. 18 illustrates block dividing in the imaging device according toEmbodiment 1 of the present invention.

FIG. 19 illustrates a calculation area for calculating parallaxevaluation values in the imaging device according to Embodiment 1 of thepresent invention.

FIG. 20 illustrates the relationship between parallax and parallaxevaluation values in the imaging device according to Embodiment 1 of thepresent invention.

FIG. 21 is a cross-sectional view showing the configuration of animaging device according to Embodiment 2 of the present invention.

FIG. 22 is a block diagram of the imaging device according to Embodiment2 of the present invention.

FIG. 23 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 2 of thepresent invention.

FIG. 24 is an external view of an intensity correction chart used ingenerating intensity correction coefficients according to Embodiment 2of the present invention.

FIGS. 25A to 25C are waveform diagrams showing an imaging signal, anintensity correction coefficient and an imaging signal after correctionin the imaging device according to Embodiment 2 of the presentinvention.

FIG. 26 is an external view of an origin correction chart used ingenerating origin correction coefficients according to Embodiment 2 ofthe present invention.

FIGS. 27A to 27D show imaging signals when images are captured of theorigin correction chart according to Embodiment 2 of the presentinvention.

FIG. 28 is an external view of a distortion correction chart used ingenerating distortion correction coefficients according to Embodiment 2of the present invention.

FIG. 29 shows an imaging signal when an image is captured of thedistortion correction chart according to Embodiment 2 of the presentinvention.

FIG. 30 is a flowchart showing a method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 2 of the presentinvention.

FIG. 31 shows coordinates referenced when generating distortioncorrection coefficients by linear interpolation.

FIGS. 32A and 32B are external views of distortion correction chartsused in generating distortion correction coefficients in a modificationof Embodiment 2 of the present invention.

FIG. 33 is a cross-sectional view showing the configuration of animaging device according to Embodiment 3 of the present invention.

FIG. 34 is a block diagram of the imaging device according to Embodiment3 of the present invention.

FIG. 35 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 3 of thepresent invention.

FIG. 36 is an external view of an intensity/origin correction chart usedin generating intensity correction coefficients and origin correctioncoefficients according to Embodiment 3 of the present invention.

FIG. 37 is a flowchart showing a method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 3 of the presentinvention.

FIG. 38 is a cross-sectional view showing the configuration of animaging device according to Embodiment 4 of the present invention.

FIG. 39 is a block diagram of the imaging device according to Embodiment4 of the present invention.

FIG. 40 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 4 of thepresent invention.

FIG. 41 is an external view of an intensity/origin/distortion correctionchart used in generating intensity correction coefficients, origincorrection coefficients, and distortion correction coefficientsaccording to Embodiment 4 of the present invention.

FIG. 42 is a flowchart showing a method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 4 of the presentinvention.

FIG. 43 is a cross-sectional view showing the configuration of animaging device according to Embodiment 5 of the present invention.

FIG. 44 is a top view of the lens of the imaging device according toEmbodiment 5 of the present invention.

FIG. 45 is a top view of the circuit unit of the imaging deviceaccording to Embodiment 5 of the present invention.

FIG. 46 is a block diagram of the imaging device according to Embodiment5 of the present invention.

FIG. 47 is a flowchart showing the operations of the imaging deviceaccording to Embodiment 5 of the present invention.

FIG. 48 is a flowchart showing a distance calculation operationaccording to Embodiment 5 of the present invention.

FIG. 49 illustrates a calculation area for calculating parallaxevaluation values in the imaging device according to Embodiment 5 of thepresent invention.

FIG. 50 is a cross-sectional view showing the configuration of aconventional imaging device.

FIG. 51 illustrates the relationship between the aperture, the lensunits, and peripheral brightness.

FIG. 52 shows the light intensity distributions for the green, red andblue components.

BEST MODE FOR CARRYING OUT THE INVENTION

An imaging device of the present invention has a plurality of lens unitseach including at least one lens, a plurality of imaging areascorresponding one-to-one with the plurality of lens units, and eachhaving a light receiving surface substantially perpendicular to theoptical axis direction of the corresponding lens unit, an imaging signalinput unit that receives as input a plurality of imaging signals eachoutput from a different one of the imaging areas, an intensitycorrection coefficient saving unit that saves an intensity correctioncoefficient, which is information concerning intensity unevenness in theimaging areas, and the intensity correcting unit that corrects theintensity of each of the plurality of imaging signals using theintensity correction coefficient, so as to reduce the effect ofintensity unevenness in the imaging areas.

When there is variability in component precision or assembly, lightintensity distribution is biased relative to the center of the opticalaxis depending on the color, producing false colors. The imaging deviceof the present invention enables biasing of light intensity distributionto be compensated by saving intensity correction coefficients andcorrecting the intensity of imaging signals based on the intensitycorrection coefficients, so as to reduce the effect of intensityunevenness in the imaging areas. The occurrence of false colors isthereby suppressed where, for example, images are synthesized fromimaging signals after correction, enabling fine images to besynthesized.

The above imaging device preferably further includes an optical elementon a light path of light incident on at least two of the plurality ofimaging areas that has transmission characteristics substantiallycentered on a first wavelength, and an optical element on a light pathof light incident on the remaining imaging areas that has transmissioncharacteristics substantially centered on a different wavelength fromthe first wavelength. Further, the intensity correcting unit preferablycorrects the intensity of at least the imaging signals corresponding tothe imaging areas, of the plurality of the imaging areas, that receivelight passing through the optical elements having transmissioncharacteristics substantially centered on the first wavelength. Notethat the first wavelength preferably is perceived as substantially greenby human vision, for example.

The above imaging device preferably further includes a parallaxcalculating unit that derives a parallax between images formed by theplurality of lens units, based on the imaging signals whose intensityhas been corrected by the intensity correcting unit, and a parallaxcorrecting unit that corrects the plurality of imaging signals andperforms image synthesis based on the parallax.

When there is variability in component precision or assembly, lightintensity distribution is biased relative to the center of the opticalaxis depending on the color, and parallax cannot be derived correctly.This preferable configuration enables correct parallax to be derivedsince biasing of light intensity distribution is compensated bycorrecting the intensity of the imaging signals, and parallax is derivedbased on the corrected imaging signals. Also, since image synthesis isperformed based on the correct parallax so as to reduce the effect ofparallax, fine images can be synthesized.

Alternatively, the above imaging device preferably further includes aparallax calculating unit that derives a parallax between images formedby the plurality of lens units, based on the imaging signals whoseintensity has been corrected by the intensity correcting unit, and adistance calculating unit that derives a distance to a subject based onthe parallax.

When there is variability in component precision or assembly, lightintensity distribution is biased relative to the center of the opticalaxis depending on the color, and parallax cannot be correctly derived.This preferable configuration enables the correct parallax to be derivedsince biasing of light intensity distribution is compensated bycorrecting the intensity of the imaging signals, and parallax is derivedbased on the corrected imaging signals. Also, the distance to thesubject can be correctly derived based on the correct parallax.

The above imaging device preferably further includes a block dividingunit that divides at least one of the plurality of imaging signals intoa plurality of blocks, and the parallax calculating unit preferablycalculates the parallax between images formed by the plurality of lensunits for each block.

According to this configuration, at least one of the plurality ofimaging signals is divided into a plurality of blocks, the intensity ofimaging signals corresponding to the at least two imaging areas thatreceive light passing through the light transmission elements having thesame wavelength characteristics is corrected, biasing of light intensitydistribution is compensated, and the correct parallax can be derived foreach block based on the corrected imaging signals. Also, since imagesynthesis is performed based on correct parallax so as to reduce theeffect of parallax for each block, fine images can be synthesized.Alternatively, the distance to the subject can be derived correctlybased on the correct parallax.

The above imaging device preferably further includes an origincorrection coefficient saving unit that saves an origin correctioncoefficient, which is information concerning correspondence between anorigin of the optical axes of the plurality of lens units and an originof the imaging signals, and an origin correcting unit that corrects anorigin of each of the plurality of imaging signals based on the origincorrection coefficient.

The above imaging device preferably further includes a distortioncorrection coefficient saving unit that saves a distortion correctioncoefficient, which is information concerning distortion of the lensunits, and a distortion correcting unit that corrects each of theplurality of imaging signals based on the distortion correctioncoefficient, so as to reduce the effect of distortion of the pluralityof lens units.

In the above imaging device, the intensity correcting unit preferablycorrects the plurality of imaging signals such that intensity levels areequal.

A imaging device manufacturing method according to the present inventionis for an imaging device that has a plurality of lens units eachincluding at least one lens, a plurality of imaging areas correspondingone-to-one with the plurality of lens units, and each having a lightreceiving surface substantially perpendicular to the optical axisdirection of the corresponding lens unit, an imaging signal input unitthat receives as input a plurality of imaging signals each output from adifferent one of the imaging areas, an intensity correction coefficientsaving unit that saves an intensity correction coefficient, which isinformation concerning intensity unevenness in the imaging areas, and anintensity correcting unit that corrects the intensity of the imagingsignals using the intensity correction coefficient, so as to reduce theeffect of intensity unevenness in the imaging areas, and includes afirst image capturing step of using the imaging device to capture animage of a substantially white object, an intensity correctioncoefficient calculating step of calculating the intensity correctioncoefficient based on an imaging signal obtained in the first imagecapturing step, and a step of saving the intensity correctioncoefficient calculated in the intensity correction coefficientcalculating step to the intensity correction coefficient saving unit.

By generating intensity correction coefficients based on imaging signalsobtained by capturing an image of a substantially white object such as adisplay or a test chart displaying uniform white light, for example, andwriting the intensity correction coefficients to an intensity correctioncoefficient saving unit in the manufacturing process, biasing of thelight intensity distribution can be compensated and the occurrence offalse colors can be suppressed even if the variability in componentprecision or assembly is different for each device, enabling fine imagesto be synthesized.

In the above manufacturing method, preferably the imaging device furtherincludes an origin correction coefficient saving unit that saves anorigin correction coefficient, which is information concerningcorrespondence between an origin of the optical axes of the plurality oflens units and an origin of the imaging signals, and an origincorrecting unit that corrects an origin of the imaging signals based onthe origin correction coefficient, and the manufacturing method furtherincludes a second image capturing step of using the imaging device tocapture an image of an object having a pattern that includes a cross ina central portion thereof, an origin correction coefficient calculatingstep of calculating the origin correction coefficient based on animaging signal obtained in the second image capturing step, and a stepof saving the origin correction coefficient calculated in the origincorrection coefficient calculating step to the origin correctioncoefficient saving unit.

Origin correction coefficients for compensating origin deviationresulting from manufacturing variability caused by deviation in the lensunits during manufacture, positional displacement of the imagingelements, or the like, thereby can be calculated and saved to an origincorrection coefficient saving unit.

In the above manufacturing method, preferably the imaging device furtherincludes a distortion correction coefficient saving unit that saves adistortion correction coefficient, which is information concerningdistortion of the lens units, and a distortion correcting unit thatcorrects the imaging signals based on the distortion correctioncoefficient, so as to reduce the effect of distortion of the pluralityof lens units, and the manufacturing method further includes a thirdimage capturing step of using the imaging device to capture an image ofan object having a lattice pattern, a distortion correction coefficientcalculating step of calculating the distortion correction coefficientbased on an imaging signal obtained in the third image capturing step,and a step of saving the distortion correction coefficient calculated inthe distortion correction coefficient calculating step to the distortioncorrection coefficient saving unit.

This manufacturing method enables distortion correction coefficients forcompensating lens distortion to be calculated and saved to a distortioncorrection coefficient saving unit.

In the above manufacturing method, preferably the imaging device furtherincludes an origin correction coefficient saving unit that saves anorigin correction coefficient, which is information concerningcorrespondence between an origin of the optical axes of the plurality oflens units and an origin of the imaging signals, and an origincorrecting unit that corrects an origin of the imaging signals based onthe origin correction coefficient, an object having a substantiallywhite background and a pattern that includes a cross in a centralportion thereof is used as the object in the first image capturing step,and the manufacturing method further includes an origin correctioncoefficient calculating step of calculating the origin correctioncoefficient based on the imaging signal obtained in the first imagecapturing step, and a step of saving the origin correction coefficientcalculated in the origin correction coefficient calculating step to theorigin correction coefficient saving unit.

This manufacturing method enables the number of times image capture isperformed in the manufacturing process to be reduced and the tact timeof the manufacturing process to be shortened, since intensity correctioncoefficients and origin correction coefficients are generated using thesame imaging signal obtained by performing image capture once.

In the above manufacturing method, preferably the imaging device furtherincludes a distortion correction coefficient saving unit that saves adistortion correction coefficient, which is information concerningdistortion of the lens units, and a distortion correcting unit thatcorrects the imaging signals based on the distortion correctioncoefficient, so as to reduce the effect of distortion of the pluralityof lens units, an object having a substantially white background and alattice pattern is used as the object in the first image capturing step,and the manufacturing method further includes a distortion correctioncoefficient calculating step of calculating the distortion correctioncoefficient based on the imaging signal obtained in the first imagecapturing step, and a step of saving the distortion correctioncoefficient calculated in the distortion correction coefficientcalculating step to the distortion correction coefficient saving unit.

This manufacturing method enables the number of times image capture isperformed in the manufacturing process to be reduced and the tact timeof the manufacturing process to be shortened, since intensity correctioncoefficients and distortion correction coefficients are generated usingthe same imaging signal obtained by performing image capture once.

In the above manufacturing method, preferably the imaging device furtherincludes an origin correction coefficient saving unit that saves anorigin correction coefficient, which is information concerningcorrespondence between an origin of the optical axes of the plurality oflens units and an origin of the imaging signals, an origin correctingunit that corrects an origin of the imaging signals based on the origincorrection coefficient, a distortion correction coefficient saving unitthat saves a distortion correction coefficient, which is informationconcerning distortion of the lens units, and a distortion correctingunit that corrects the imaging signals based on the distortioncorrection coefficient, so as to reduce the effect of distortion of theplurality of lens units, an object having a substantially whitebackground and a lattice pattern is used as the object in the firstimage capturing step, and the manufacturing method further includes anorigin correction coefficient calculating step of calculating the origincorrection coefficient based on the imaging signal obtained in the firstimage capturing step, a distortion correction coefficient calculatingstep of calculating the distortion correction coefficient based on theimaging signal obtained in the first image capturing step, and a step ofsaving the origin correction coefficient calculated in the origincorrection coefficient calculating step to the origin correctioncoefficient saving unit, and saving the distortion correctioncoefficient calculated in the distortion correction coefficientcalculating step to the distortion correction coefficient saving unit.

This manufacturing method enables the number of times image capture isperformed in the manufacturing process to be reduced and the tact timeof the manufacturing process to be shortened, since intensity correctioncoefficients, origin correction coefficients and distortion correctioncoefficients are generated using the same imaging signal obtained byperforming image capture once.

Hereinafter, specific embodiments of the present invention will bedescribed with reference to the drawings.

Embodiment 1

An imaging device according to Embodiment 1 of the present inventionsaves intensity correction coefficients, and corrects the intensity ofimaging signals based on the intensity correction coefficients such thatthe degree of correction changes depending on the position of theimaging area. Biasing of light intensity distribution is therebycompensated, the occurrence of false colors is suppressed, and fineimages are synthesized.

The imaging device according to Embodiment 1 of the present inventiondivides at least one of the plurality of imaging signals into aplurality of blocks, corrects the intensity of imaging signalscorresponding to the at least two imaging areas that receive lightpassing through the light transmission elements having the samewavelength characteristics, compensates for biasing of light intensitydistribution, derives a parallax for each block based on the correctedimaging signals, and performs image synthesis based on this parallax soas to reduce the effect of parallax for each block. Since biasing oflight intensity distribution is thereby compensated, correct parallaxderived, and image synthesis performed based on this correct parallax,fine images can be synthesized.

The imaging device according to Embodiment 1 of the present inventionsaves an origin correction coefficient, corrects the origin of imagingsignals based on the origin correction coefficient, derives a parallaxfor each block based on the corrected imaging signals, and performsimage synthesis based on this parallax so as to reduce the effect ofparallax for each block. Since origin deviation is thereby compensated,correct parallax derived, and image synthesis performed based on thiscorrect parallax, fine images can be synthesized.

The imaging device according to Embodiment 1 of the present inventionsaves a distortion correction coefficient, corrects imaging signalsbased on the distortion correction coefficient so as to reduce theeffect of distortion of the plurality of lens units, derives a parallaxfor each block based on the corrected imaging signals, and performsimage synthesis based on this parallax so as to reduce the effect ofparallax for each block. Since the effect of distortion is therebyreduced, correct parallax is derived, and image synthesis is performedbased on this correct parallax, fine images can be synthesized.

The imaging device according to Embodiment 1 of the present inventionwill be described with reference to the drawings.

FIG. 1 is a cross-sectional view showing the configuration of theimaging device according to Embodiment 1 of the present invention. InFIG. 1, an imaging device 101 has a lens module unit 110 and a circuitunit 120.

The lens module unit 110 has a lens barrel 111, an upper cover glass112, a lens 113, a fixed actuator portion 114, and a movable actuatorportion 115. The circuit unit 120 has a substrate 121, a package 122, animaging element 123, a package cover glass 124, and a system LSI(hereinafter, SLSI) 125.

The lens barrel 111 is cylindrical and formed by injection-moldingresin, and the inner surface thereof is lusterless black in order toprevent diffused reflection of light. The upper cover glass 112 isdiscoid, formed from transparent resin, and anchored to the top surfaceof the lens barrel 111 using adhesive or the like, and the surfacethereof is provided with a protective film for preventing damage causedby friction or the like and an antireflective film for preventingreflection of incident light.

FIG. 2 is a top view of the lens 113 of the imaging device according toEmbodiment 1 of the present invention. The lens 113 is substantiallydiscoid and formed from glass or transparent resin, and has a first lensunit 113 a, a second lens unit 113 b, a third lens unit 113 c, and afourth lens unit 113 d disposed in a grid. The X-axis and the Y-axis areset as shown in FIG. 2, along the directions in which the first tofourth lens units 113 a to 113 d are disposed. Light incident on thefirst lens unit 113 a, the second lens unit 113 b, the third lens unit113 c, and the fourth lens unit 113 d from the side on which the subjectis positioned is emitted to the side on which the imaging element 123 ispositioned, and four images are formed on the imaging element 123.

The fixed actuator portion 114 is anchored to the inner surface of thelens barrel 111 by adhesive or the like. The movable actuator portion115 is anchored to the outer periphery of the lens 113 by adhesive orthe like. The fixed actuator portion 114 and the movable actuatorportion 115 constitute a voice coil motor. The fixed actuator portion114 has a permanent magnet (not shown) and a ferromagnetic yoke (notshown), while the movable actuator portion 115 has a coil (not shown).The movable actuator portion 115 is elastically supported by an elasticbody (not shown) relative to the fixed actuator portion 114. The movableactuator portion 115 moves relative to the fixed actuator portion 114 asa result of energizing the coil of the movable actuator portion 115,which changes the relative distance along the optical axis between thelens 113 an the imaging element 123.

The substrate 121 is constituted by a resin substrate, and is anchoredby adhesive or the like, with the bottom surface of the lens barrel 111contacting the top thereof. The circuit unit 120 is thus anchored to thelens module unit 110 to constitute the imaging device 101.

The package 122 is formed from resin having a metal terminal, and isanchored inside the lens barrel 111 by soldering or the like the metalterminal unit to the top surface of the substrate 121. The imagingelement 123 is constituted by a first imaging element 123 a, a secondimaging element 123 b, a third imaging element 123 c, and a fourthimaging element 123 d. The first imaging element 123 a, the secondimaging element 123 b, the third imaging element 123 c and the fourthimaging element 123 d are solid state imaging elements such as CCDsensors or CMOS sensors, and are disposed such that the centers of thelight receiving surfaces thereof are substantially aligned with thecenters of the optical axes of the first lens unit 113 a, the secondlens unit 113 b, the third lens unit 113 c and the fourth lens unit 113d, and such that the light receiving surfaces of the imaging elementsare substantially perpendicular to the optical axes of the correspondinglens units. The terminals of the first imaging element 123 a, the secondimaging element 123 b, the third imaging element 123 c and the fourthimaging element 123 d are connected with gold wires 127 by wire bondingto the metal terminal on a bottom portion of the package 122 on theinside thereof, and electrically connected to the SLSI 125 via thesubstrate 121. Light emitted from the first lens unit 113 a, the secondlens unit 113 b, the third lens unit 113 c and the fourth lens unit 113d forms images on the light receiving surfaces of the first imagingelement 123 a, the second imaging element 123 b, the third imagingelement 123 c and the fourth imaging element 123 d, and electricalinformation converted from optical information by a photodiode is outputto the SLSI 125.

FIG. 3 is a top view of the circuit unit 120 of the imaging deviceaccording to Embodiment 1 of the present invention. The package coverglass 124 is flat, formed using transparent resin, and anchored to thetop surface of the package 122 by adhesive or the like. A first colorfilter 124 a, a second color filter 124 b, a third color filter 124 c, afourth color filter 124 d and a shading portion 124 e are disposed onthe top surface of the package cover glass 124 by vapor deposition orthe like. An infrared blocking filter (not shown; hereinafter, IRfilter) is provided on the bottom surface of the package cover glass 124by vapor deposition or the like.

FIG. 4 is a characteristics diagram of the color filters of the imagingdevice according to Embodiment 1 of the present invention, while FIG. 5is a characteristics diagram of the IR filter of the imaging deviceaccording to Embodiment 1 of the present invention. The first colorfilter 124 a has a spectral transmission characteristic of transmittingmainly green as shown by G in FIG. 4, the second color filter 124 b hasa spectral transmission characteristic of transmitting mainly blue asshown by B in FIG. 4, the third color filter 124 c has a spectraltransmission characteristic of transmitting mainly red as shown by R inFIG. 4, and the fourth color filter has spectral transmissioncharacteristics for transmitting mainly green as shown by G in FIG. 4.The IR filter has a spectral transmission characteristic of blockinginfrared light as shown by IR in FIG. 5.

Consequently, object light incident from a top portion of the first lensunit 113 a is emitted from a bottom portion of the first lens unit 113a, and the first imaging element 123 a receives the green component ofthe object light, since mainly green is transmitted by the first colorfilter 124 a and IR filter, and forms an image on the light receivingportion of the first imaging element 123 a. Object light incident from atop portion of the second lens unit 113 b is emitted from a bottomportion of the second lens unit 113 b, and the second imaging element123 b receives the blue component of the object light, since mainly blueis transmitted by the second color filter 124 b and IR filter, and formsan image on the light receiving portion of the second imaging element123 b. Object light incident from a top portion of the third lens unit113 c is emitted from a bottom portion of the third lens unit 113 c, andthe third imaging element 123 c receives the red component of the objectlight since mainly red is transmitted by the third color filter 124 cand IR filter, and forms an image on the light receiving portion of thethird imaging element 123 c. Further, object light incident from a topportion of the fourth lens unit 113 d is emitted from a bottom portionof the fourth lens unit 113 d, and the fourth imaging element 123 dreceives the green component of the object light since mainly green istransmitted by the fourth color filter 124 d and IR filter, and forms animage on the light receiving portion of the fourth imaging element 123d.

The SLSI 125 controls the energizing of the coil of the movable actuatorportion 115, drives the imaging element 123, receives as inputelectrical information from the imaging element 123, performs variousimage processing, communicates with a host CPU, and outputs imagesexternally as described later.

The relationship between subject distance and parallax will be describednext. Since the camera module according to Embodiment 1 of the presentinvention has four lens units (first lens unit 113 a, second lens unit113 b, third lens unit 113 c, fourth lens unit 113 d), the relativeposition of the four object images respectively formed by the four lensunits changes according to subject distance.

FIG. 6 illustrates the position of images of an object at infinity inthe imaging device according to Embodiment 1 of the present invention.In FIG. 6, only the first lens unit 113 a, the first imaging element 123a, the second lens unit 113 b, and the second imaging element 123 b areshown for simplicity. Since light L1 of light from an object 10 atinfinity incident on the first lens unit 113 a is parallel with light L2incident on the second lens unit 113 b, the distance between the firstlens unit 113 a and the second lens unit 113 b is equal to the distancebetween an object image 11 a on the first imaging element 123 a and anobject image 11 b on the second imaging element 123 b. Here, the opticalaxis of the first lens unit 113 a, the optical axis of the second lensunit 113 b, the optical axis of the third lens unit 113 c, and theoptical axis of the fourth lens unit 113 d are disposed so as tosubstantially coincide respectively with the centers of the lightreceiving surfaces of the first imaging element 123 a, the center of thesecond imaging element 123 b, the center of the third imaging element123 c, and the center of the fourth imaging element 123 d. Consequently,the relative positional relation of the centers of the light receivingsurfaces of the first imaging element 123 a, the second imaging element123 b, the third imaging element 123 c and the fourth imaging element123 d with the images of the object at infinity respectively formed onthe light receiving surfaces is the same for all of the imagingelements. In other words, there is no parallax.

FIG. 7 illustrates the position of images of an object at a finitedistance in the imaging device according to Embodiment 1 of the presentinvention. In FIG. 7, only the first lens unit 113 a, the first imagingelement 123 a, the second lens unit 113 b, and the second imagingelement 123 b are shown for simplicity. Since light L1 of light from anobject 12 at a finite distance incident on the first lens unit 113 a isnot parallel with light L2 incident on the second lens unit 113 b, thedistance between an object image 13 a on the first imaging element 123 aand an object image 13 b on the second imaging element 123 b is longerthan the distance between the first lens unit 113 a and the second lensunit 113 b. Thus, the relative positional relationship of the centers ofthe light receiving surfaces of the first imaging element 123 a, thesecond imaging element 123 b, the third imaging element 123 c and thefourth imaging element 123 d with the images of the object at a finitedistance respectively formed on the light receiving surfaces differs foreach imaging elements. In other words, there is parallax. The parallax Ais expressed by the following equation (1), given that the righttriangle whose two sides A and D in FIG. 7 form a right angle is similarto the right triangle whose two sides f and Δ form a right angle, whereA is the distance to the object image 12 (subject distance), D is thedistance between the first lens unit 113 a and the second lens unit 113b, and f is the focal length of the lens units 113 a and 113 b. Notethat the asterisk “*” in the following equation (1) and in otherequations described below denotes the multiplication operator. A similarrelation is established between the other lens units. The relativepositions of the four object images respectively formed by the four lensunits 113 a, 113 b, 113 c and 113 d thus change according to the subjectdistance. For example, the parallax A increases with decreases in thesubject distance A.

Δ=f*D/A  (1)

The relationship between contrast and focal length will be describednext.

FIG. 8A illustrates the relationship between an in-focus (focused) imageand a contrast evaluation value in the imaging device according toEmbodiment 1 of the present invention, and FIG. 8B illustrates therelationship between an out-of-focus (not focused) image and a contrastevaluation value in the imaging device according to Embodiment 1 of thepresent invention. The figures on the left-hand side in FIG. 8A and FIG.8B are images captured of a rectangle whose left half is white and righthalf is black. As shown in the figure on the left-hand side of FIG. 8A,the outline of the captured image when in-focus is distinct and contrastis high. On the other hand, as shown in the figure on the left-hand sideof FIG. 8B, the outline of the captured image when out-of-focus isblurred and contrast is low. The figures on the right-hand side of FIG.8A and FIG. 8B show the results when a band-pass filter (BPF) is used onthe information signals in the figures on the left-hand side. Thehorizontal axis plots the position in the x axis direction and thevertical axis plots output values after band-pass filtering. The signalamplitude after band-pass filtering when in-focus is large, as shown inthe figure on the right-hand side of FIG. 8A, while the signal amplitudeafter band-pass filtering when out-of-focus is small, as shown in thefigure on the right-hand side of FIG. 8B. Here, the signal amplitudeafter band-pass filtering is defined as a contrast evaluation valueshowing the level of contrast. Thus, the contrast evaluation value ishigh when in-focus, as shown in the figure on the right-hand side ofFIG. 8A, and the contrast evaluation value is low when out-of-focus, asshown in the figure on the right-hand side of FIG. 8B.

FIG. 9 illustrates the relationship between the lens position and thecontrast evaluation value in the imaging device according to Embodiment1 of the present invention. The contrast evaluation value will be smallwhen the distance between the lens 113 and the imaging element 123 isshort (z1) when capturing an image of a given object, because the objectwill not be in focus. The contrast evaluation value gradually increasesas the distance between the lens 113 and the imaging element 123gradually increases, and is maximized when in-focus (z2). Further, theobject goes out of focus and the contrast evaluation value decreases asthe distance between the lens 113 and the imaging element 123 graduallyincreases (z3). The contrast evaluation value is thus maximized whenin-focus.

The operations of the imaging device according to Embodiment 1 of thepresent invention will be described next. FIG. 10 is a block diagram ofthe imaging device according to Embodiment 1 of the present invention.The SLSI 125 has a system control unit 131, an imaging element driveunit 132, an imaging signal input unit 133, an actuator manipulatedvariable output unit 134, an image processing unit 135, an input/outputunit 136, an intensity correction coefficient memory 137, an origincorrection coefficient memory 138, and a distortion correction memory139. The circuit unit 120 has an amplifier 126 in addition to the aboveconfiguration.

The amplifier 126 applies a voltage that depends on the output from theactuator manipulated variable output unit 134 to the coil of the movableactuator portion 115.

The system control unit 131, which is constituted by a CPU (centralprocessing unit), a memory and the like, controls the overall SLSI 125.

The imaging element drive unit 132, which is constituted by a logiccircuit and the like, generates a signal for driving the imaging element123, and applies a voltage that depends on this signal to the imagingelement 123.

The imaging signal input unit 133 is constituted by a first imagingsignal input unit 133 a, a second imaging signal input unit 133 b, athird imaging signal input unit 133 c, and a fourth imaging signal inputunit 133 d. The first imaging signal input unit 133 a, the secondimaging signal input unit 133 b, the third imaging signal input unit 133c, and the fourth imaging signal input unit 133 d, each of which isconfigured with a CDS circuit (correlated double sampling circuit), anAGC (automatic gain controller), and an ADC (analog digital converter)connected in series, and are respectively connected to the first imagingelement 123 a, the second imaging element 123 b, the third imagingelement 123 c, and the fourth imaging element 123 d, receives as inputelectrical signals from the imaging elements, remove static noise usingthe CDS circuit, adjust gains using the AGC, convert the analog signalsto digital values using the ADC, and write the digital values to thememory of the system control unit 131.

The actuator manipulated variable output unit 134, which is constitutedby a DAC (digital analog converter), outputs a voltage signal thatdepends on the voltage to be applied to the coil of the movable actuatorportion 115.

The image processing unit 135, which is configured to include a logiccircuit or a DSP (digital signal processor), or both, performs variousimage processing in accordance with prescribed program controls, usinginformation in the memory of the system control unit 131. The imageprocessing unit 135 has an autofocus control unit 141, an intensitycorrecting unit 142, an origin correcting unit 143, a distortioncorrecting unit 144, and a parallax correcting unit 145.

The input/output unit 136 communicates with the host CPU (not shown),and outputs image signals to the host CPU, an external memory (notshown) and an external display device such as an LCD (not shown).

The intensity correction coefficient memory 137, which is constituted bya nonvolatile memory such as a flash memory or a FeRAM (ferroelectricrandom access memory), saves intensity correction coefficients for useby the intensity correcting unit 142. The origin correction coefficientmemory 138, which is constituted by a nonvolatile memory such as a flashmemory or a FeRAM, saves origin correction coefficients for use by theorigin correcting unit 143. The distortion correction coefficient memory139, which is constituted by a nonvolatile memory such as a flash memoryor a FeRAM, saves distortion correction coefficients for use by thedistortion correcting unit 144.

FIG. 11 is a flowchart showing the operations of the imaging deviceaccording to Embodiment 1 of the present invention. The imaging device101 is operated by the system control unit 131 of the SLSI 125 as perthis flowchart.

In step S1000, operations are started. For example, the imaging device101 starts operations as the result of the host CPU (not shown)detecting that a shutter button (not shown) has been pressed, andinstructing the imaging device 101 to start operations via theinput/output unit 136. Step S1100 is executed next.

In step S1100, the autofocus control unit 141 executes autofocuscontrols. FIG. 12 is a flowchart showing autofocus control operationsaccording to Embodiment 1 of the present invention. The flowchart ofFIG. 12 shows the operations of step S1100 in detail.

In step S1110, the autofocus control operations are started. Step S1121is executed next.

In step S1121, a counter i is initiated to 0. Step S1122 is executednext.

In step S1122, the position command for the actuator is calculated. Aposition command Xact for the actuator is calculated using the counteri, as in the following equation (2). Note that the position command Xactindicates the position at which the sense towards the subject ispositive, based on the in-focus position of the image at infinity. Here,kx is the set value. Step S1123 is executed next.

Xact=kx*i  (2)

In step S1123, the actuator manipulated variable (voltage applied to thecoil of the movable actuator portion 115) Vact is calculated using amanipulated variable function shown by the following equation (3). Here,ka and kb are the respectively set values. Step S1124 is executed next.

Vact=ka*Xact+kb  (3)

In step S1124, the actuator is operated. The actuator manipulatedvariable output unit 134 changes the output voltage signal such that thevoltage applied to the coil (not shown) of the movable actuator portion115 after passing through the amplifier 126 will be Vact. Step S1125 isexecuted next.

In step S1125, the subject image incident on the first lens unit 113 aand formed on the first imaging element 123 a is captured. The imagingelement drive unit 132 outputs signals for operating an electronicshutter and/or performing transfer as needed, as a result ofinstructions from the system control unit 131. The first imaging signalinput unit 133 a, in sync with signals generated by the imaging elementdrive unit 132, receives as input an imaging signal, which is an analogsignal of an image output by the first imaging element 123 a, removesstatic noise using the CDS, automatically adjusts input gain using theAGC, converts the analog signal to a digital value using the ADC, andwrites the digital value to the memory of a prescribed address in thesystem control unit 131 as a first imaging signal I1(x, y). FIG. 13illustrates the coordinates of an imaging signal of the imaging deviceaccording to Embodiment 1 of the present invention. I1(x, y) indicatesthe first imaging signal of the x-th horizontal and y-th vertical pixel.The total number of pixels is H×L, where H is the number of pixels inthe height direction and L is the number of pixels in the lengthdirection of the input image, with x changing from 0 to L−1, and ychanging from 0 to H−1. Step S1126 is executed next.

In step S1126, an autofocus control block is set. A rectangular area ina vicinity of the center of the image area is assumed to be theautofocus control block. Note that this block need not necessarily be ina vicinity of the center of the image area, and may be set to reflectthe intentions of the user operating the camera, for instance (e.g.,detecting the view direction with a sensor). Note also that a pluralityof blocks may be selected rather than a single block, and the average ofthe contrast evaluation values for use in autofocus control (describedhereinafter) in the plurality of blocks may be used. Also, the contrastevaluation values for use in autofocus control (described hereinafter)may be calculated for a plurality of blocks, and at least one of theblocks later selected as the autofocus control block. Step S1127 isexecuted next.

In step S1127, a contrast evaluation value for use in autofocus controlis generated using data in the memory of the system control unit 131.This calculation is performed for pixels in the autofocus control blockof the first imaging signal I1. The absolute values of the Laplacian,which is the sum of the second-order derivatives of the x and ydirections, are calculated as in the following equation (11), spatiallyfiltered using a LPF (low-pass filter) as in the following equation(12), and averaged in the autofocus control block to obtain an autofocuscontrol contrast evaluation value C3 as in the following equation (13).Here, Naf is the number of pixel in the autofocus control block. StepS1128 is executed next.

$\begin{matrix}{{C\; 1\left( {x,y} \right)} = {{{I\; 1\left( {{x - 1},y} \right)} + {I\; 1\left( {{x + 1},y} \right)} + {I\; 1\left( {x,{y - 1}} \right)} + {I\; 1\left( {x,{y + 1}} \right)} - {4I\; 1\left( {x,y} \right)}}}} & (11) \\{{C\; 2\left( {x,y} \right)} = {{C\; 1\left( {{x - 1},{y - 1}} \right)} + {C\; 1\left( {x,{y - 1}} \right)} + {C\; 1\left( {{x + 1},{y - 1}} \right)} + {C\; 1\left( {{x - 1},y} \right)} + {C\; 1\left( {x,y} \right)} + {C\; 1\left( {{x + 1},y} \right)} + {C\; 1\left( {{x - 1},{y + 1}} \right)} + {C\; 1\left( {x,{y + 1}} \right)} + {C\; 1\left( {{x + 1},{y + 1}} \right)}}} & (12) \\{{C\; 3} = {\sum{C\; 2{\left( {x,y} \right)/{Naf}}}}} & (13)\end{matrix}$

In step S1128, the contrast evaluation value C3 is written to the memoryof the system control unit 131 as C3(i), as in the following equation(14). Step S1129 is executed next.

C3(i)=C3  (14)

In step S1129, 1 is added to the counter i as in the following equation(15). Step S1130 is executed next.

i=i+1  (15)

In step S1130, the counter i is compared with a threshold Saf, andbranching is performed depending on the result. If the counter i issmaller than the threshold Saf (comparison result of step S1130=Y), stepS1122 is executed next. On the other hand, if the counter i is greaterthan or equal to the threshold Saf (comparison result of step S1130=N),step S1140 is executed next. The processing from step S1122 to stepS1128 is thus repeated Saf number of times by initiating the counter ito 0 in step S1121, adding 1 to the counter i in step S1129, andperforming branching with the counter i in step S1130.

In step S1140, the contrast evaluation value C3 is evaluated. As in FIG.9, the contrast evaluation value C3 is maximized at the in-focusposition. The counter value i that gives this maximum value is assumedto be a counter value iaf that gives the maximum contrast, as in thefollowing equation (16). Step S1151 is executed next.

iaf=i giving maximum value of C3   (16)

In step S1151, the position command for the actuator is calculated. Theposition command Xact for the actuator is calculated using the countervalue iaf giving the maximum contrast, as in the following equation(17). Note that position command Xact indicates the position at whichthe sense towards the subject is positive based on the in-focus positionof the image at infinity. Step S1152 is executed next.

Xact=kx*iaf  (17)

In step S1152, the actuator manipulated variable (voltage applied to thecoil of the movable actuator portion 115) Vact is calculated using amanipulated variable function. The description of this operation, whichis similar to step S1123, is omitted. Step S1153 is executed next.

In step S1153, the actuator is operated. The description of thisoperation, which is similar to step S1124, is omitted. Step S1160 isexecuted next.

In step S1160, autofocus control is ended and processing returns to themain routine. Accordingly, step S1200 of FIG. 11 is executed next.

In step S1200, an image is input. The imaging element drive unit 132outputs signals for operating an electronic shutter and/or performingtransfer as needed, as a result of instructions from the system controlunit 131. The first imaging signal input unit 133 a, the second imagingsignal input unit 133 b, the third imaging signal input unit 133 c, andthe fourth imaging signal input unit 133 d, in sync with signalsgenerated by the imaging element drive unit 132, respectively receive asinput imaging signals, which are analog signals of images output by thefirst imaging element 123 a, the second imaging element 123 b, the thirdimaging element 123 c and the fourth imaging element 123 d, removestatic noise using the CDS, automatically adjust input gains using theAGC, converts the analog signals to digital values using the ADC, andwrite the digital values to the memory of prescribed addresses in thesystem control unit 131 as a first imaging signal I1(x, y), a secondimaging signal I2(x, y), a third imaging signal I3(x, y), and a fourthimaging signal I4(x, y). FIG. 13 illustrates the coordinates of animaging signal of the camera module according to Embodiment 1 of thepresent invention. I1(x, y) indicates the first imaging signal of thex-th horizontal and y-th vertical pixel. The total number of pixels isH×L, where H is the number of pixels in the height direction and L isthe number of pixels in the length direction of the input image, with xchanging from 0 to L−1, and y changing from 0 to H−1. The second imagingsignal I2(x, y), the third imaging signal I3(x, y), and the fourthimaging signal I4(x, y) are similar. That is, I2(x, y), I3(x, y) andI4(x, y) respectively show the second imaging signal, the third imagingsignal and the fourth imaging signal of the x-th horizontal and y-thvertical pixel. The respective total number of pixels is H×L, where H isthe number of pixels in the height direction and L is the number ofpixels in the length direction of the input image, with x changing from0 to L−1, and y changing from 0 to H−1. Step S1300 is executed next.

In step S1300, the intensity correcting unit 142 corrects the firstimaging signal I1, the second imaging signal I2, the third imagingsignal I3, and the fourth imaging signal I4 using intensity correctioncoefficients saved in the intensity correction coefficient memory 137.The results are then written to the memory of the system control unit131.

A first intensity correction coefficient a1(x, y) for use in correctingthe first imaging signal I1(x, y), a second intensity correctioncoefficient a2(x, y) for use in correcting the second imaging signalI2(x, y), a third intensity correction coefficient a3(x, y) for use incorrecting the third imaging signal I3(x, y), and a fourth intensitycorrection coefficient a4(x, y) for use in correcting the fourth imagingsignal I4(x, y) are saved in the intensity correction coefficient memory137. a1(x, y), a2(x, y), a3(x, y) and a4(x, y) respectively indicate thefirst intensity correction coefficient, the second intensity correctioncoefficient, the third intensity correction coefficient and the fourthintensity correction coefficient of the x-th horizontal and y-thvertical pixel. The total number of pixels is H×L, where H is the numberof pixels in the height direction and L is the number of pixels in thelength direction of the input image, with x changing from 0 to L−1, andy changing from 0 to H−1.

FIG. 14 is a flowchart showing the intensity correction operationaccording to Embodiment 1 of the present invention. The flowchart ofFIG. 14 shows step S1300, in which intensity correction is performed, indetail.

Firstly, in step S1320, a correction value for each pixel (x, y) iscalculated. The results of respectively multiplying kab1, kab2, kab3 andkab4 by the first intensity correction coefficient a1(x, y), the secondintensity correction coefficient a2(x, y), the third intensitycorrection coefficient a3(x, y), and the fourth intensity correctioncoefficient a4(x, y) are set as a first intensity correction value b1(x,y), a second intensity correction value b2(x, y), a third intensitycorrection value b3(x, y), and a fourth intensity correction value b4(x,y), as in the following equations (18), (19), (20), and (21). Step S1330is executed next.

b1(x,y)=kab1*a1(x,y)  (18)

b2(x,y)=kab2*a2(x,y)  (19)

b3(x,y)=kab3*a3(x,y)  (20)

b4(x,y)=kab4*a4(x,y)  (21)

In step S1330, intensity correction is performed. Intensity is correctedby respectively multiplying the first imaging signal I1(x, y), thesecond imaging signal I2(x, y), the third imaging signal I3(x, y), andthe fourth imaging signal I4(x, y) by the first intensity correctionvalue b1(x, y), the second intensity correction value b2(x, y), thethird intensity correction value b3(x, y), and the fourth intensitycorrection value b4(x, y), as in the following equations (22), (23),(24), and (25). Step S1340 is executed next.

I1(x,y)=I1(x,y)*b1(x,y)  (22)

I2(x,y)=I2(x,y)*b2(x,y)  (23)

I3(x,y)=I3(x,y)*b3(x,y)  (24)

I4(x,y)=I4(x,y)*b4(x,y)  (25)

In step S1340, intensity correction is ended and processing returns tothe main routine. Accordingly, step S1400 of FIG. 11 is executed next.

In step S1400, the origin correcting unit 143 corrects the first imagingsignal I1, the second imaging signal I2, the third imaging signal I3,and the fourth imaging signal I4 using origin correction coefficientssaved in the origin correction coefficient memory 138. The results arethen written to the memory of the system control unit 131.

A first origin correction coefficient g1 x, g1 y for use in correctingthe first imaging signal I1(x, y), a second origin correctioncoefficient g2 x, g2 y for use in correcting the second imaging signalI2(x, y), a third origin correction coefficient g3 x, g3 y for use incorrecting the third imaging signal I3(x, y), and a fourth origincorrection coefficient g4 x, g4 y for use in correcting the fourthimaging signal I4(x, y) are saved in the origin correction coefficientmemory 138. g1 x, g2 x, g3 x and g4 x respectively indicate the xcomponent of the origin correction coefficients, and g1 y, g2 y, g3 yand g4 y respectively indicate the y component of the origin correctioncoefficients.

The first imaging signal I1 is corrected so as to be moved by the firstorigin correction coefficient (g1 x, g1 y), as in the following equation(26). The second imaging signal I2 is corrected so as to be moved by thesecond origin correction coefficient (g2 x, g2 y), as in the followingequation (27). The third imaging signal I3 is corrected so as to bemoved by the third origin correction coefficient (g3 x, g3 y), as in thefollowing equation (28). The fourth imaging signal I4 is corrected so asto be moved by the fourth origin correction coefficient (g4 x, g4 y), asin the following equation (29). Note that if the values on the rightside of equations (26), (27), (28) and (29) do not exist (e.g., x=0,y=0, g1 x and g1 y are positive numbers), the same value as an existingneighboring value, a value inferred from neighbors by extrapolation orthe like, or zero or the like may be substituted. Note that g1 x, g2 x,g3 x, g4 x, g1 y, g2 y, g3 y and g4 y are not limited to being integers,and may be decimals. In this case, the nearest neighboring value or avalue interpolated from neighboring pixels is used as the right side ofequations (26), (27), (28) and (29). Step S1500 of FIG. 11 is executednext.

I1(x,y)=I1(x−g1x,y−g1y)  (26)

I2(x,y)=I2(x−g1x,y−g1y)  (27)

I3(x,y)=I3(x−g1x,y−g1y)  (28)

I4(x,y)=I4(x−g1x,y−g1y)  (29)

In step S1500, the distortion correcting unit 144 corrects the firstimaging signal I1, the second imaging signal I2, the third imagingsignal I3, and the fourth imaging signal I4 using distortion correctioncoefficients saved in the distortion correction coefficient memory 138.The results are then written to the memory of the system control unit131.

A first distortion correction coefficient p1 x(x, y), p1 y(x, y) for usein correcting the first imaging signal I1(x, y), a second distortioncorrection coefficient p2 x(x, y), p2 y(x, y) for use in correcting thesecond imaging signal I2(x, y), a third distortion correctioncoefficient p3 x(x, y), p3 y(x, y) for use in correcting the thirdimaging signal I3(x, y), and a fourth distortion correction coefficientp4 x(x, y), p4 y(x, y) for use in correcting the fourth imaging signal14(x, y) are saved in the distortion correction coefficient memory 139.p1 x(x, y), p2 x(x, y), p3 x(x, y) and p4 x(x, y) respectively indicatethe x component of the first distortion correction coefficient, thesecond distortion correction coefficient, the third distortioncorrection coefficient, and the fourth distortion correction coefficientof the x-th horizontal and y-th vertical pixel. p1 y(x, y), p2 y(x, y),p3 y(x, y) and p4 y(x, y) respectively indicate the y component of thefirst distortion correction coefficient, the second distortioncorrection coefficient, the third distortion correction coefficient, andthe fourth distortion correction coefficient of the x-th horizontal andy-th vertical pixel. The total number of pixels is H×L, where H is thenumber of pixels in the height direction and L is the number of pixelsin the length direction of the input image, with x changing from 0 toL−1, and y changing from 0 to H−1.

FIG. 15 illustrates distortion correction coefficients according toEmbodiment 1 of the present invention. The intersections of the latticeof dotted lines indicate ideal coordinates, while the intersections ofthe lattice of solid lines indicate the distorted coordinates. As shownin FIG. 15, an image that should ideally be imaged at (0, 0), forexample, is actually imaged at a position that deviates by (p1 x(0, 0),p1 y(0, 0)) from (0, 0) due to distortion. The amount of deviation (p1x(0, 0), p1 y(0, 0)) is assumed to be the distortion correctioncoefficient, and stored in the distortion correction memory 139. Thatis, an image that should ideally be imaged at (x, y) is actually imagedat a position that deviates by (p1 x(x, y), p1 y(x, y)) from (x, y) dueto distortion. The amount of deviation (p1 x(x, y), p1 y(x, y)) isassumed to be the distortion correction coefficient, and stored in thedistortion correction memory 139.

FIG. 16 is a flowchart showing the distortion correction operationaccording to Embodiment 1 of the present invention. The flowchart ofFIG. 16 shows step S1500, in which distortion correction is performed,in detail.

Firstly, in step S1520, distortion correction coordinates for each pixel(x, y) are calculated. The results of adding the value x of the xcoordinate to the x component p1 x(x, y) of the first distortioncorrection coefficient, the x component p2 x(x, y) of the seconddistortion correction coefficient, the x component p3 x(x, y) of thethird distortion correction coefficient, and the x component p4 x(x, y)of the fourth distortion correction coefficient are set as the xcomponent q1 x(x, y) of the first distortion correction coordinate, thex component q2 x(x, y) of the second distortion correction coordinate,the x component q3 x(x, y) of the third distortion correctioncoordinate, and the x component q4 x(x, y) of the fourth distortioncorrection coordinate, as in the following equations (30), (31), (32)and (33). The results of adding the value y of the y coordinate to the ycomponent p1 y(x, y) of the first distortion correction coefficient, they component p2 y(x, y) of the second distortion correction coefficient,the y component p3 y(x, y) of the third distortion correctioncoefficient, and the y component p4 y(x, y) of the fourth distortioncorrection coefficient are set as the y component q1 y(x, y) of thefirst distortion correction coordinate, the y component q2 y(x, y) ofthe second distortion correction coordinate, the y component q3 y(x, y)of the third distortion correction coordinate, and the y component q4y(x, y) of the fourth distortion correction coordinate, as in thefollowing equations (34), (35), (36) and (37). Step S1530 is executednext.

q1x(x,y)=x+p1x(x,y)  (30)

q2x(x,y)=x+p2x(x,y)  (31)

q3x(x,y)=x+p3x(x,y)  (32)

q4x(x,y)=x+p4x(x,y)  (33)

q1y(x,y)=y+p1y(x,y)  (34)

q2y(x,y)=y+p2y(x,y)  (35)

q3y(x,y)=y+p3y(x,y)  (36)

q4y(x,y)=y+p4y(x,y)  (37)

In step S1330, distortion correction is performed. A first imagingsignal I1(q1 x(x, y), q1 y(x, y)) on first distortion correctioncoordinates (q1 x(x, y), q1 y(x, y)), a second imaging signal I2(q2 x(x,y), q2 y(x, y)) on second distortion correction coordinates (q2 x(x, y),q2 y(x, y)), a third imaging signal 13(q3 x(x, y), q3 y(x, y)) on thirddistortion correction coordinates (q3 x(x, y), q3 y(x, y)), and a fourthimaging signal I4(q4 x(x, y), q4 y(x, y)) on fourth distortioncorrection coordinates (q4 x(x, y), q4 y(x, y)) are used as the firstimaging signal I1(x, y), the second imaging signal I2(x, y), the thirdimaging signal I3(x, y), and the fourth imaging signal I4(x, y) on thecoordinates (x, y), as in the following equations (38), (39), (40) and(41). Note that if the values on the right side of equations (38), (39),(40) and (41) do not exist, the same value as an existing neighboringvalue, a value inferred from neighbors by extrapolation or the like, orzero or the like may be substituted. Note that q1 x(x, y), q2 x(x, y),q3 x(x, y), q4 x(x, y), q1 y(x, y), q2 y(x, y), q3 y(x, y) and q4 y(x,y) are not limited to being integers, and may be decimals. In this case,the nearest neighboring value or a value interpolated from neighboringpixels is used as the right side of equations (38), (39), (40) and (41).Step S1540 is executed next.

I1(x,y)=I1(q1x(x,y),q1y(x,y))  (38)

I2(x,y)=I2(q2x(x,y),q2y(x,y))  (39)

I3(x,y)=I3(q3x(x,y),q3y(x,y))  (40)

I4(x,y)=I4(q4x(x,y),q4y(x,y))  (41)

In step S1540, distortion correction is ended and processing returns tothe main routine. Accordingly, step S1600 of FIG. 11 is executed next.

In step S1600, parallax correction is performed. FIG. 17 is a flowchartshowing the parallax correction operation according to Embodiment 1 ofthe present invention. The flowchart of FIG. 17 shows the operations ofstep S1600 in detail.

Firstly, in step S1620, block dividing is performed. FIG. 18 illustratesblock dividing in the imaging device according to Embodiment 1 of thepresent invention. As shown in FIG. 18, the first imaging signal I1 isdivided into M blocks in the length direction and N blocks in the heightdirection, giving a total of M×N blocks, with the respective blocksshown by B_(i). Here, i changes from 0 to M×N−1. Step S1630 is executednext.

In step S1630, a parallax value is calculated for each block. Firstly, aparallax evaluation value (R_(0(k)), R_(1(k)), . . . , R_(i(k)), . . . ,R_(MN-1(k)), k=0, 1, . . . , kmax) is calculated for each block (B₀, B₁,. . . , B_(i), . . . , B_(MN-1)). FIG. 19 illustrates a calculation areafor calculating parallax evaluation values in the imaging deviceaccording to Embodiment 1 of the present invention. The area shown byB_(i) (also shown as I1) is the i-th block derived at step S1620 fromthe first imaging signal I1. The area shown by I4 is an area in whichB_(i) has been moved by k in the x direction and k in the y direction.The total sum of absolute differences shown by the following expression(42) is then calculated as a parallax evaluation value R_(i(k)) for allimage signals I1(x, y) and I4(x−k, y−k) of the respective areas. Here,ΣΣ shows the total sum of all pixels in the block B_(i).

R _(i(k)) =|I1(x,y)−I4(x−k,y−k)|  (42)

This parallax evaluation value R_(i(k)) shows the level of correlationbetween the first image signal I1 of the i-th block B_(i) and the fourthimage signal I4 in an area removed by (k, k) in the x and y directions,respectively. The smaller the value, the greater the correlation(similarity). FIG. 20 illustrates the relationship between parallax andparallax evaluation values in the imaging device according to Embodiment1 of the present invention. As shown in FIG. 20, the parallax evaluationvalue R_(i(k)) changes depending on the value of k, and is minimizedwhen k=Δi. This shows that the image signal I1 of the block is mostclosely correlated to (most closely resembles) the fourth image signalI4 obtained by moving the i-th block B_(i) of the first image signal I1by (−Δi, −Δi) in the x and y directions, respectively. Consequently, weknow that the parallax in the x and y directions between the firstimaging signal I1 and the fourth imaging signal I4 in relation to i-thblock B_(i) is (Δi, Δi). Hereinafter, this Δi will be called theparallax Δi of the i-th block B_(i). The parallax Δi of B_(i) is thusderived from i=0 to i=M×N−1. Step S1640 is executed next.

In step S1640, parallax correction and image synthesis are performed.The result of this is written to the memory of the system control unit131. Since the first imaging element 123 a and the fourth imagingelement 123 d mainly receive the green component of object light, thefirst imaging signal I1 and the fourth imaging signal I4 are theinformation signals of the green component of object light. Also, sincethe second imaging element 123 b mainly receives the blue component ofobject light, the second imaging signal 12 is the information signal ofthe blue component of object light. Further, since the third imagingelement 123 c mainly receives the red component of object light, thethird imaging signal I3 is the information signal of the red componentof object light. Since the parallax between the first imaging element123 a and the fourth imaging element 123 d in relation to the i-th blockB_(i) is predicted to be (Δi, Δi), G(x, y) showing the intensity ofgreen at the pixel coordinates (x, y) are assumed to be the average ofthe first imaging signal I1(x, y) and the fourth imaging signal I4(x−Δi,y−Δi), as in the following expression (43). Taking the average in thisway enables the effect of random noise to be reduced. Also, since theparallax between the first imaging element 123 a and the second imagingelement 123 b is predicted to be (Δi, 0), B(x, y) showing the intensityof blue at the pixel coordinates (x, y) are assumed to be the secondimaging signal I2(x−Δi, y), as in the following expression (44).Further, since the parallax between the first imaging element 123 a andthe third imaging element 123 c is predicted to be (0, Δi), R(x, y)showing the intensity of red at the pixel coordinates (x, y) are assumedto be the third imaging signal I3(x, y−Δi), as in the followingexpression (45).

Step S1650 is executed next.

G(x,y)=[I1(x,y)+I4(x−Δi,y−Δi)]/2  (43)

B(x,y)=I2(x−Δi,y)  (44)

R(x,y)=I3(x,y−Δi)  (45)

In step S1650, parallax correction is ended and processing returns tothe main routine. Accordingly, step S1700 of FIG. 11 is executed next.

In step S1700, image output is performed. The input/output unit 136outputs G(x, y), B(x, y) and R(x, y), which are pieces of data in thememory of the system control unit 131, to the host CPU (not shown) or anexternal display device (not shown). Note that output such as luminanceor color difference signals, for example, may be output instead of G(x,y), B(x, y) and R(x, y). Values after image processing such as whitebalance or y (gamma) correction may also be output. Further, dataobtained by performing lossless compression or lossy compression such asJPEG may be output. A plurality of these may also be output. Step S1800is executed next.

In step S1800, operations are ended.

As a result of being configured and operated as above, the imagingdevice 101 has the following effects.

When there is variability in component precision or assembly, lightintensity distribution is biased relative to the center of the opticalaxis depending on the color, producing false colors. With the imagingdevice of Embodiment 1, the intensity correction coefficients a1, a2, a3and a4 are saved to the intensity correction coefficient memory 137,which is a nonvolatile memory, and in step S1300, the intensitycorrection values b1(x, y), b2(x, y), b3(x, y) and b4(x, y), whosedegree of correction changes depending on the position (x, y) of theimaging area, are generated based on the intensity correctioncoefficients a1, a2, a3 and a4, and the imaging signals I1(x, y), I2(x,y), I3(x, y) and I4(x, y) are corrected. Biasing of light intensitydistribution is thereby compensated and the occurrence of false colorsis suppressed, enabling fine images to be synthesized.

Also, the imaging device of Embodiment 1 generates the intensitycorrection values b1(x, y), b2(x, y), b3(x, y) and b4(x, y), whosedegree of correction changes depending on the position (x, y) of theimaging area, based on the intensity correction coefficients a1, a2, a3and a4, corrects the imaging signals I1(x, y), I2(x, y), I3(x, y) andI4(x, y), compensates for biasing of light intensity distribution instep S1300, divides the first imaging signal I1 into a plurality ofblocks in step S1620, derives a parallax for each block based on thecorrected imaging signals I1(x, y) and I4(x, y) in step S1630, andperforms image synthesis based on these parallaxes in step S1640, so asto reduce the effect of parallax for each block. Since biasing of lightintensity distribution is thereby compensated, correct parallax derived,and image synthesis performed based on this correct parallax, fineimages can be synthesized.

The imaging device of Embodiment 1 saves the origin correctioncoefficients g1 x, g2 x, g3 x, g4 x, g1 y, g2 y, g3 y and g4 y to theorigin correction coefficient memory 138, corrects the origins of theimaging signals I1(x, y), I2(x, y), I3(x, y) and I4(x, y) based on theorigin correction coefficients g1 x, g2 x, g3 x, g4 x, g1 y, g2 y, g3 yand g4 y in step S1400, derives a parallax for each block based on thecorrected imaging signals I1(x, y) and I4(x, y) in step S1630, andperforms image synthesis based on these parallaxes in step S1640, so asto reduce the effect of parallax for each block. Since correct parallaxis thereby derived, and image synthesis is performed based on thiscorrect parallax, fine images can be synthesized.

The imaging device of Embodiment 1 saves the distortion correctioncoefficients p1 x(x, y), p2 x(x, y), p3 x(x, y), p4 x(x, y), p1 y(x, y),p2 y(x, y), p3 y(x, y) and p4 y(x, y), calculates the distortioncorrection coordinates q1 x(x, y), q2 x(x, y), q3 x(x, y), q4 x(x, y),q1 y(x, y), q2 y(x, y), q3 y(x, y), and q4 y(x, y), based on thedistortion correction coefficients p1 x(x, y), p2 x(x, y), p3 x(x, y),p4 x(x, y), p1 y(x, y), p2 y(x, y), p3 y(x, y) and p4 y(x, y) in stepS1520, corrects the imaging signals I1(x, y), I2(x, y), I3(x, y) andI4(x, y) at the distortion correction coordinates q1 x(x, y), q2 x(x,y), q3 x(x, y), q4 x(x, y), q1 y(x, y), q2 y(x, y), q3 y(x, y), and q4y(x, y) in step S1530, so as to reduce the effect of distortion of theplurality of lens units, derives a parallax for each block based on thecorrected imaging signals I1(x, y) and I4(x, y) in step S1630, andperforms image synthesis based on these parallaxes in step S1640, so asto reduce the effect of parallax for each block. Since correct parallaxis derived and image synthesis is performed based on this correctparallax, fine images can be synthesized.

Note that the imaging device of Embodiment 1 also has the effect ofsuppressing the occurrence of false colors, even where biasing of thesensitivity of the first imaging element 123 a, the second imagingelement 123 b, the third imaging element 123 c, and the fourth imagingelement 123 d respectively differ.

Note that with the imaging device of Embodiment 1, the calculatedparallaxes are used without modification, although they may beappropriately limited. Depending on the lens characteristics, imageswill lack sharpness when the subject distance A is less than a givenvalue. Thus, the maximum value of the parallax Δ will be decided if thisvalue is set as the minimum value of the subject distance A. Parallaxgreater than this value may be disregarded as an error. In such cases,the parallax evaluation value may also employ the second smallest valueas the parallax.

With the imaging device of Embodiment 1, parallax is calculated from thefirst imaging signal I1 (mainly showing green) and the fourth imagingsignal I4 (mainly showing green), although the present invention is notlimited to this. Since a purple subject, for example, has little greencomponent and includes plenty of blue component and red component, theremay be times when parallax cannot be calculated from the first imagingsignal I1 (mainly showing green) and the fourth imaging signal I4(mainly showing green). In this case, parallax may be calculated fromthe second imaging signal I2 (mainly showing blue) and the third imagingsignal I3 (mainly showing red). If parallax cannot be calculated fromthe first parallax signal I1 (mainly showing green) and the fourthparallax signal I4 (mainly showing green), or from the second imagingsignal I2 (mainly showing blue) and the third imaging signal I3 (mainlyshowing red), it is assumed that there is no parallax effect, andprocessing can be performed as if there is no parallax.

In the imaging device of Embodiment 1, an IR filter is used on lightthat passes through all of the lens units, although part of the IRfilter may be omitted, and not used on light passing through some of thelens units. The IR filter may also be completely omitted.

In the imaging device of Embodiment 1, the first color filter 124 a, thesecond color filter 124 b, the third color filter 124 c and the fourthcolor filter 124 d respectively transmit mainly green, blue, red andgreen, although the wavelengths may differ. Using complementary colors,for example, the first color filter 124 a, the second color filter 124b, the third color filter 124 c and the fourth color filter 124 d mayrespectively transmit mainly yellow, cyan, magenta and yellow. Further,the order may be interchanged. For example, the first color filter 124a, the second color filter 124 b, the third color filter 124 c and thefourth color filter 124 d may respectively transmit mainly green, green,blue and red. Alternatively, the first color filter 124 a, the secondcolor filter 124 b, the third color filter 124 c and the fourth colorfilter 124 d may respectively transmit mainly red, blue, green and red.

By disposing the first to fourth imaging elements 123 a to 123 d suchthat the second imaging element 123 b is disposed so as to be at the topand the third imaging element 123 c is disposed so as to be at thebottom during image capture when the imaging device of Embodiment 1 ismounted in a camera, the upper side will be sensitive to blue and thelower side will be sensitive to red, enabling colors to be reproducedmore naturally in landscape photographs.

When two extremums are marked in the parallax evaluation value, thelarger of the parallaxes is employed. Since a subject and a backgroundare included in these blocks, and the subject distance differs from thebackground distance, two extremums appear. Because the subject distanceis small in comparison to the background distance, the parallax of thesubject is large in comparison to the parallax of the background. Here,although the effect of the parallax of the background cannot be reducedif the larger of the parallaxes is employed, the effect of the parallaxof the subject, which directly affects image quality, can be reduced.

The image output timing is not limited to the above, and preview outputmay be appropriately performed. For example, an image that has notundergone parallax correction may be output during the autofocus controlin step S1100.

In Embodiment 1, the imaging element 123 is constituted by the firstimaging element 123 a, the second imaging element 123 b, the thirdimaging element 123 c, and the fourth imaging element 123 d, and theimaging signal input unit 133 is constituted by the first imaging signalinput unit 133 a, the second imaging signal input unit 133 b, the thirdimaging signal input unit 133 c, and the fourth imaging signal inputunit 133 d. However, the imaging element 123 may be constituted by asingle imaging element, and four images may be formed by the first tofourth lens units 113 a to 113 d at different positions on the lightreceiving surface thereof. The imaging signal input unit 133 may beconstituted from a single imaging signal input unit that receives asinput signals from the single imaging element 123. In this case, an areais appropriately selected from data placed in the memory of the systemcontrol unit 131, and the selected data is set as the first imagingsignal I1, the second imaging signal I2, the third imaging signal I3,and the fourth imaging signal I4.

In the first embodiment, the first lens unit 113 a, the second lens unit113 b, the third lens 113 c, and the fourth lens unit 113 d are disposedsuch that a rectangle obtained by joining together the centers of theoptical axes thereof is square, although the present invention is notlimited to this. The lengths of the rectangle in the x and y directionsmay differ. In this case, appropriate changes will be necessary, forinstance, when deriving parallaxes in step S1630, or when correctingparallaxes in step S1640. That is, k is changed so as to maintain theratio between lengths in the x and y directions of the above rectangle,rather than using the same value of k for the x and y directions.

Note that although the above description illustrates the configurationand operations of a device that performs various corrections on imagingsignals obtained through image capture and corrects parallax beforesynthesizing images from the imaging signals, the imaging device of thepresent invention can also be applied as a measuring device fordetecting distance to the subject. That is, the imaging device of thepresent invention can also be implemented as a device that calculatesdistance based on parallax obtained as aforementioned, and outputs theobtained distance, with practical application as a surveying device,inter-vehicular distance detecting device or the like being conceivable.That is, equation (1), when solved for distance A, is as shown in thefollowing equation (46). Accordingly, the distance to the subject fromthe block B_(i) is as calculated in the following equation (47), and thedistance to the subject from a pixel (x, y) included in the block B_(i)is as shown in the following equation (48), and saved in the memory ofthe system control unit 131. Note that the units of measurement arechanged appropriately when the calculations are performed. If thedistance information A(x, y) is then output externally via theinput/output unit 136, an imaging device that functions as a measuringdevice for detecting distance can be realized.

A=f*D/Δ  (46)

A _(i) =f*D/Δ _(i)  (47)

A(x,y)=A_(i)((x,y) included in B_(i))  (48)

Embodiment 2

An imaging device according to Embodiment 2 of the present inventiongenerates intensity correction coefficients in the manufacturingprocess, and writes the intensity correction coefficients to anintensity correction coefficient memory. Biasing of light intensitydistribution is thereby compensated and the occurrence of false colorsis suppressed, allowing fine images to be synthesized, even if thevariability in component precision or assembly is different for eachdevice.

Further, the imaging device according to Embodiment 2 of the presentinvention divides at least one of the imaging signals into a pluralityof blocks, generates intensity correction coefficients in themanufacturing process, and writes the intensity correction coefficientsto an intensity correction coefficient memory. Biasing of lightintensity distribution is thereby compensated and a parallax is derivedfor each block based on the corrected imaging signals, enabling imagesynthesis to be performed based on these parallaxes so as to reduce theeffect of parallax for each block, even if the variability in componentprecision or assembly is different for each device. Since biasing oflight intensity distribution is thereby compensated, correct parallaxderived, and image synthesis performed based on this correct parallax,fine images can be synthesized.

The imaging device according to Embodiment 2 of the present inventiongenerates origin correction coefficients in the manufacturing process,and writes the origin correction coefficients to an origin correctioncoefficient memory. Variability in origin is thereby compensated, and aparallax is derived for each block based on the corrected imagingsignals, enabling image synthesis to be performed based on theseparallaxes so as to reduce the effect of parallax for each block, evenif the variability in component precision or assembly is different foreach device. Since origin deviation is thereby compensated, correctparallax derived, and image synthesis performed based on this correctparallax, fine images can be synthesized.

The imaging device according to Embodiment 2 of the present inventiongenerates distortion correction coefficients in the manufacturingprocess, and writes the distortion correction coefficients to adistortion correction coefficient memory. Imaging signals are therebycorrected so as to reduce the effect of distortion of the plurality oflens units, and a parallax is derived for each block based on thecorrected imaging signals, enabling image synthesis to be performedbased on these parallaxes so as to reduce the effect of parallax foreach block, even if the variability in component precision or assemblyis different for each device. Since the effect of distortion is therebyreduced, correct parallax is derived, and image synthesis is performedbased on this correct parallax, fine images can be synthesized.

The imaging device according to Embodiment 2 of the present inventionwill be described with reference to the drawings. FIG. 21 is across-sectional view showing the configuration of the imaging deviceaccording to Embodiment 2 of the present invention. In FIG. 21, animaging device 201 has a lens module unit 110 and a circuit unit 220.

The lens module unit 110 has a lens barrel 111, an upper cover glass112, a lens 113, a fixed actuator portion 114, and a movable actuatorportion 115. The circuit unit 220 has a substrate 121, a package 122, animaging element 123, a package cover glass 124, and a system LSI(hereinafter, SLSI) 225. The configurations and operations apart fromthe SLSI 225 are similar to Embodiment 1, with the same referencenumerals attached and redundant description omitted.

FIG. 22 is a block diagram of the imaging device according to Embodiment2 of the present invention. The SLSI 225 has a system control unit 231,an imaging element drive unit 132, an imaging signal input unit 133, anactuator manipulated variable output unit 134, an image processing unit135, an input/output unit 136, an intensity correction coefficientmemory 137, an origin correction coefficient memory 138, a distortioncorrection coefficient memory 139, an intensity correction coefficientgenerating unit 251, an origin correction coefficient generating unit252, and a distortion correction coefficient generating unit 253. Thecircuit unit 220 has an amplifier 126 in addition to the aboveconfiguration.

In an inspection process during the manufacturing process after assemblyof the imaging device 201, the intensity correction coefficientgenerating unit 251 generates intensity correction coefficients a1(x,y), a2(x, y), a3(x, y), a4(x, y), and writes the intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y), a4(x, y) to the intensitycorrection coefficient memory 137, as described hereinafter. Also, theorigin correction coefficient generating unit 252 generates origincorrection coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y,and writes the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y to the origin correction coefficient memory 138.Further, the distortion correction coefficient generating unit 253generates distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y), andwrites the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139.

FIG. 23 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 2 of thepresent invention. 201 is the imaging device according to Embodiment 2of the present invention, 202 is a correction coefficient generationcontroller, 203 is a display, and 204 is a cable for connecting theimaging device. The correction coefficient generation controller 202,which is constituted by a computer, performs coordinated control of theimaging device 201 and the display 204, and causes the imaging device201 to generate the intensity correction coefficients a1(x, y), a2(x,y), a3(x, y), a4(x, y), the origin correction coefficients g1 x, g1 y,g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and the distortion correctioncoefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y),p3 y(x, y), p4 x(x, y), p4 y(x, y). The display 203, which isconstituted by a CRT (cathode ray tube) display or the like, iscontrolled by the correction coefficient generation controller 202,operates in coordination with the imaging device 201, and draws images(charts) used in generating the intensity correction coefficients a1(x,y), a2(x, y), a3(x, y), a4(x, y), the origin correction coefficients g1x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y). The imaging device cable204, which is constituted by a USB (universal serial bus) cable or thelike, connects the imaging device 201 and the correction coefficientgeneration controller 202, feeds power to the imaging device 201, and isthe medium through which communication is performed between the imagingdevice 201 and the correction coefficient generation controller 202.

The operations of the imaging device according to Embodiment 2 of thepresent invention will be described next. FIG. 24 is an external view ofan intensity correction chart used in generating intensity correctioncoefficients according to Embodiment 2 of the present invention. Theimaging device of Embodiment 2 captures an image of an intensitycorrection chart 205, which is a uniform white subject such as in FIG.24, in the manufacturing process. FIGS. 25A to 25C are waveform diagramsshowing an imaging signal, an intensity correction coefficient and animaging signal after correction in the imaging device according toEmbodiment 2 of the present invention. In FIGS. 25A to 25C, only thefirst imaging signal I1, the first intensity correction coefficient a1and the first imaging signal I1 after correction are shown forsimplicity. As shown in FIG. 25A, this imaging signal I1 is intensityinformation concerning biasing of light intensity distribution,including intensity information on reductions in peripheral brightness.The intensity correction coefficient generating unit 251 derives thereciprocal of the imaging signal I1 as the intensity correctioncoefficient a1 such as shown in FIG. 25B, and saves the result in theintensity correction coefficient memory 137. That is, the intensitycorrection coefficient generating unit 251 derives a1(x, y), such thatI1(x, y)*a1(x, y)=1. When intensity correction is performed as shown instep S1300, the imaging signal after correction will be flat as shown inFIG. 25C, and a uniform subject image will be reproduced. Similarly, theintensity correction coefficient generating unit 251 generates thesecond intensity correction coefficient a2 for the second imaging signalI2, the third intensity correction coefficient a3 for the third imagingsignal I3, and the fourth intensity correction coefficient a4 for thefourth imaging signal I4, and writes the generated intensity correctioncoefficients to the intensity correction coefficient memory 137. Here,when there is variability in component precision or assembly, lightintensity distribution is biased relative to the center of the opticalaxis depending on the color, and reductions in peripheral brightnessrespectively differ, the second intensity correction coefficient a2, thethird intensity correction coefficient a3, and the fourth intensitycorrection coefficient a4 will respectively differ. Capturing an imageof a uniform white subject and setting the reciprocal of the resultantimaging signal as the intensity correction coefficient thus enablesintensity correction to be appropriately performed. The imaging deviceof Embodiment 2 uses the intensity correction coefficient generatingunit 251 to generate intensity correction coefficients based on thisprinciple, and writes the generated intensity correction coefficients tothe intensity correction coefficient memory 137. The intensitycorrecting unit 142 of the image processing unit 135 then performsintensity correction based on the intensity correction coefficientssaved in the intensity correction coefficient memory 137.

FIG. 26 is an external view of an origin correction chart used ingenerating origin correction coefficients according to Embodiment 2 ofthe present invention. The imaging device of Embodiment 2 captures animage of an origin correction chart 206, which is a uniform whitesubject with a cross drawn thereon such as in FIG. 26, in themanufacturing process. The imaging device 201 is disposed so as todirectly oppose the origin correction chart 206, and the centers of theoptical axes of the plurality of lens of the imaging device 201 aredisposed so as to coincide with the center of the cross in the origincorrection chart 206. FIGS. 27A to 27D show imaging signals of theorigin correction chart 206 when images are captured according toEmbodiment 2 of the present invention. By capturing an image of theorigin correction chart 206, signals such as shown in FIGS. 27A to 27Dare obtained as the first imaging signal I1, the second imaging signalI2, the third imaging signal I3 and the fourth imaging signal I4. Thecross-shaped solid lines in FIGS. 27A to 27D are images captured of thecross drawn on the origin correction chart 206. Note that thecross-shaped broken lines in FIGS. 27A to 27D hypothetically show linespassing through the centers of the images in order to facilitatecomparison with the imaging signals, and are not included in the actualsignals. As shown in FIGS. 27A to 27D, the centers of the crosses(centers of the solid lines) of the first imaging signal I1, the secondimaging signal I2, the third imaging signal I3, and the fourth imagingsignal I4 respectively deviate by (s1 x, s1 y), (s2 x, s2 y), (s3 x, s3y) and (s4 x, s4 y) in comparison with the centers of the images(centers of the broken lines). This deviation is caused by a combinationof parallax resulting from the origin correction chart 206 beingdisposed at a finite distance from the imaging device 201 andmanufacturing variability resulting from deviation of lens units duringmanufacture, positional deviation of the imaging element or the like.Consequently, subtracting the coefficients contributing to parallax fromthe respective deviations (s1 x, s1 y), (s2 x, s2 y), (s3 x, s3 y) and(s4 x, s4 y) from the centers of the crosses gives the deviationresulting from manufacturing variability. The origin correctioncoefficient generating unit 252 generates the origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y so as toeliminate the effect of this deviation, and writes the generated origincorrection coefficients to the origin correction coefficient memory 138.The origin correcting unit 143 of the image processing unit 135 performsorigin correction in step S1400 using the origin correction coefficientsin the origin correction coefficient memory 138.

FIG. 28 is an external view of a distortion correction chart used ingenerating distortion correction coefficients according to Embodiment 2of the present invention. The imaging device of Embodiment 2 captures animage of a distortion correction chart 207, which is a uniform whitesubject with a lattice drawn thereon such as shown in FIG. 28, in themanufacturing process. The intervals in the lattice preferably equate to10 to 15 pixels in the captured image. The imaging device 201 isdisposed so as to directly oppose the distortion correction chart 207,and the centers of the optical axes of the plurality of lens of theimaging device 201 are disposed so as to coincide with the center of thelattice in the distortion correction chart 207. FIG. 29 shows an imagingsignal of the distortion correction chart 207 when an image is capturedaccording to Embodiment 2 of the present invention. In FIG. 29, only thefirst imaging signal I1 is shown for simplicity. By capturing an imageof the distortion correction chart 207, a signal such as shown in FIG.29 is obtained as the first imaging signal I1. The distorted,lattice-shaped solid lines in FIG. 29 result from capturing an image ofthe lattice drawn on the distortion correction chart 207. Note that theundistorted, lattice-shaped broken lines in FIG. 29 hypothetically showan image captured when there is no lens distortion for facilitating thecomparison with the imaging signal, and are not included in the actualimaging signal. Note that in FIG. 29, origin deviation and deviationresulting from parallax has been omitted. The intersections in thedistorted lattice of the imaging signal I1 (intersections of solidlines) are given by (ux1(i, j), uy1(i, j)), while the intersections inthe lattice when there is no lens distortion (intersections of brokenlines) are given by (vx1(i, j), vy1(i, j)). Here, (i, j) shows the i-thand j-th intersection in the x and y directions, respectively. As shownin FIG. 29, the lattice has WX+1 intersections in the x directionincluding the edges, and WY+1 intersections in the y direction includingthe edges. The upper-left intersection is given by (vx1(0, 0), vy1(0,0)), while the bottom-right intersection is given by (vx1(WX, WY),vy1(WX, WY)).

The distortion correction coefficient generating unit 253 generates adistortion correction coefficient p1 x(x, y), p1 y(x, y), so as to usethe imaging signal I1 of the coordinates (ux1(i, j), uy1(i, j)) as theimaging signal I1 of the coordinates (vx1(i, j), vy1(i, j)) afterdistortion correction. In relation to the coordinates of pixels otherthan at intersections, the distortion correction coefficient generatingunit 253 generates a distortion correction coefficient p1 x(x, y), p1y(x, y), so as to use the imaging signal I1 of coordinates derived byinterpolation from a neighboring intersection. Note that the distortioncorrection coefficient generating unit 253 also similarly generatesdistortion correction coefficients p2 x(x, y), p2 y(x, y), p3 x(x, y),p3 y(x, y) and p4 x(x, y), p4 y(x, y) in relation to the other imagingsignals I2, I3 and I4. The generated distortion correction coefficientsare stored in the distortion correction coefficient memory 139. Notethat origin deviation and deviation resulting from parallax arecorrected appropriately.

FIG. 30 is a flowchart showing a method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 2 of the presentinvention.

In step S2000, the imaging device 201 is disposed such that the centersof the optical axes of the plurality of lens units of the imaging device201 coincide with the center of the drawing area on the display 203, andthe generation of intensity correction coefficients, origin correctioncoefficients and distortion correction coefficients is started. StepS2110 is executed next.

In step S2110, the imaging device cable 204 is connected to the imagingdevice 201. This is performed by an industrial robot or the like. Poweris fed from the correction coefficient generation controller 202 to theimaging device 201, making communication possible between the correctioncoefficient generation controller 202 and the imaging device 201. StepS2120 is executed next.

In step S2120, a correction coefficient generation program is downloadedto the imaging device 201. The correction coefficient generationcontroller 202 transmits the correction coefficient generation programto the imaging device 201 after detecting that the imaging device cable204 is connected to the imaging device 201. The imaging device 201receives the correction coefficient generation program, writes thereceived program to the memory of the system control unit 131, andthereinafter proceeds to generate intensity correction coefficients inaccordance with this correction coefficient generation program. That is,while the intensity correction coefficient generating unit 251, theorigin correction coefficient generating unit 252 and the distortioncorrection coefficient generating unit 253 are illustrated in FIG. 22 asindependent blocks relative to the system control unit 231, they arehypothetical blocks in which their functions are realized as a result ofthe system control unit 131 executing the correction coefficientgeneration program. Note that since this correction coefficientgeneration program is unnecessary and thus deleted after the correctioncoefficient have been generated, the program may be saved to a volatilememory or a nonvolatile memory. Consequently, the intensity correctioncoefficient generating unit 251, the origin correction coefficientgenerating unit 252 and the distortion correction coefficient generatingunit 253 do not necessarily exist in the shipped imaging device 201.Step S2210 is executed next.

In step S2210, the correction coefficient generation controller 202causes the intensity correction chart 205, which is uniform white light,to be drawn on the display 203. Step S2220 is executed next.

In step S2220, the correction coefficient generation controller 202transmits a command to start image capture to the imaging device 201 viathe imaging device cable 204. Step S2230 is executed next.

In step S2230, the imaging device 201 captures an image of the intensitycorrection chart 205. The imaging device 201 executes this step S2230 inresponse to the command in step S2220. Description of this operation,which is similar to step S1200, is omitted. The imaging device 201 thensaves the imaging signals to the memory of the system control unit 231as a first imaging signal I1(0, x, y), a second imaging signal I2(0, x,y), a third imaging signal I3(0, x, y), and a fourth imaging signalI4(0, x, y) for use in generating intensity correction coefficients.Note that an area for saving moving images or the like in normal usageis used as this memory. Step S2310 is executed next.

In step S2310, the correction coefficient generation controller 202causes the origin correction chart 206, which has a cross disposed onbackground of uniform white light, to be drawn on the display 203. Here,the correction coefficient generation controller 202 causes the origincorrection chart 206 to be drawn such that the center of the drawingarea on the display 203, that is, the center of each optical axis of theplurality of lens of the imaging device 201, coincides with theintersection of the cross. Step S2320 is executed next.

In step S2320, the correction coefficient generation controller 202transmits a command to start image capture to the imaging device 201 viathe imaging device cable 204. Step S2330 is executed next.

In step S2330, the imaging device 201 captures an image of the origincorrection chart 206. The imaging device 201 executes this step S2330 inresponse to the command in step S2320. Description of this operation,which is similar to step S1200, is omitted. The imaging device 201 thensaves the imaging signals to the memory of the system control unit 231as a first imaging signal I1(1, x, y), a second imaging signal I2(1, x,y), a third imaging signal I3(1, x, y) and a fourth imaging signal I4(1,x, y) for use in generating origin correction coefficients. Note that anarea for saving moving images or the like in normal usage is used asthis memory. Step S2410 is executed next.

In step S2410, the correction coefficient generation controller 202causes the distortion correction chart 207, which has a lattice disposedon a background of uniform white light, to be drawn on the display 203.Here, the correction coefficient generation controller 202 causes thedistortion correction chart 207 to be drawn such that the center of thedrawing area on the display 203, that is, the center of each opticalaxis of the plurality of lens units of the imaging device 201 coincideswith the center of the lattice. Step S2420 is executed next.

In step S2420, the correction coefficient generation controller 202transmits a command to start image capture to the imaging device 201 viathe imaging device cable 204. Step S2430 is executed next.

In step S2430, the imaging device 201 captures an image of thedistortion correction chart 207. The imaging device 201 executes thisstep S2430 in response to the command in step S2320. Description of thisoperation, which is similar to step S1200, is omitted. The imagingdevice 201 then saves the imaging signals to the memory of the systemcontrol unit 231 as a first imaging signal I1(2, x, y), a second imagingsignal I2(2, x, y), a third imaging signal I3(2, x, y), and a fourthimaging signal I4(2, x, y) for use in generating distortion correctioncoefficients. Note that an area for saving moving images or the like innormal usage is used as this memory. Step S2510 is executed next.

In step S2510, the intensity correction coefficient generating unit 251generates the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y) and a4(x, y), using the first imaging signal I1(0, x, y), thesecond imaging signal I2(0, x, y), the third imaging signal I3(0, x, y),and the fourth imaging signal I4(0, x, y) captured and saved to memoryin step S2230. The intensity correction coefficient generating unit 251sets the ratio between the first imaging signal I1(0, L/2, H/2) for usein intensity correction coefficient generation positioned at the centerof the image and the first imaging signal I1(0, x, y) for use inintensity correction coefficient generation as the first intensitycorrection coefficient a1(x, y), as in the following equation (49).Here, I1(0, L/2, H/2) expresses the signal of the pixel positioned atthe center of the image (i.e., pixel whose x coordinate is L/2 and ycoordinate is H/2), out of the first imaging signals for use ingenerating intensity correction coefficients. Note that H is the numberof pixels in the height direction and L is the number of pixels in thelength direction of the image. The intensity correction coefficientgenerating unit 251 also sets the ratio between the second imagingsignal I2(0, L/2, H/2) for use in intensity correction coefficientgeneration positioned at the center of the image and the second imagingsignal I2(0, x, y) for use in intensity correction coefficientgeneration as the second intensity correction coefficient a2(x, y), asin the following equation (50), sets the ratio between the third imagingsignal I3(0, L/2, H/2) for use in intensity correction coefficientgeneration positioned at the center of the image and the third imagingsignal I3(i, x, y) for use in intensity correction coefficientgeneration as the third intensity correction coefficient a3(x, y), as inthe following equation (51), and sets the ratio between the fourthimaging signal I4(0, L/2, H/2) for use in intensity correctioncoefficient generation positioned at the center of the image and thefourth imaging signal I4(i, x, y) for use in intensity correctioncoefficient generation as the fourth intensity correction coefficienta4(x, y), as in the following equation (52). Note that image capture maybe performed a plurality of times on the intensity correction chart 205in step S2230, and an image obtained by averaging these images may beused as the imaging signal for use in generating intensity correctioncoefficients. In this case, the effect of random noise and the like canbe reduced by averaging. Step S2520 is executed next.

a1(x,y)=I1(0,L/2,H/2)/I1(0,x,y)  (49)

a2(x,y)=I2(0,L/2,H/2)/I2(0,x,y)  (50)

a3(x,y)=I3(0,L/2,H/2)/I3(0,x,y)  (51)

a4(x,y)=I4(0,L/2,H/2)/I4(0,x,y)  (52)

In step S2520, the intensity correction coefficient generating unit 251writes the intensity correction coefficients a1(x, y), a2(x, y), a3(x,y) and a4(x, y) to the intensity correction coefficient memory 137.These intensity correction coefficients a1(x, y), a2(x, y), a3(x, y) anda4(x, y) are used in the intensity correction of S1300. Step S2610 isexecuted next.

In step S2610, the origin correction coefficient generating unit 252generates the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y. The respective deviations (s1 x, s1 y), (s2 x, s2y), (s3 x, s3 y) and (s4 x, s4 y) from the centers of the crosses of thefirst imaging signal I1(1, x, y) for use in origin correctioncoefficient generation, the second imaging signal I2(1, x, y) for use inorigin correction coefficient generation, the third imaging signal I3(1,x, y) for use in origin correction coefficient generation, and thefourth imaging signal I4(1, x, y) for use in origin correctioncoefficient generation (centers of solid lines shown in FIG. 27A-27D)relative to the centers of the images (centers of broken lines shown inFIG. 27A-27D), as shown in FIG. 27, are detected. Here, methods ofderiving the coordinates of the centers of the crosses in the respectiveimaging signals I1(1, x, y), I2(1, x, y), I3(1, x, y) and I4(1, x, y)for use in origin correction coefficient generation involve, forinstance, pattern matching thumbnails of the crosses with the imagingsignals I1(1, x, y), I2(1, x, y), I3(1, x, y) and I4(1, x, y), orderiving cross patterns by binarizing the imaging signals I1(1, x, y),I2(1, x, y), I3(1, x, y) and I4(1, x, y). Subtracting the coordinates ofthe centers of the images from the coordinates of the centers of thecrosses obtained from the imaging signals I1(1, x, y), I2(1, x, y),I3(1, x, y) and I4(1, x, y) as a result of such methods gives the abovedeviations (s1 x, s1 y), (s2 x, s2 y), (s3 x, s3 y) and (s4 x, s4 y).

Next, in order to remove the effect of parallax, the origin correctioncoefficient generating unit 252 firstly calculates the amount of theparallax. The origin correction coefficient generating unit 252 derivesthe x component Δx of the parallax and the y component Δy of theparallax, as in the following equations (53) and (54). Here, f is thefocal distance of the lens, and Dx is the x component of the distancebetween the optical axes of the lens units. Dx is the distance betweenthe optical axis of the first lens 113 a and the optical axis of thesecond lens 113 b, or the distance between the optical axis of the thirdlens 113 c and the optical axis of the fourth lens 113 d, thesedistances being substantially equal. Dy is the y component of thedistance between the optical axes of the lens units. Dy is the distancebetween the optical axis of the first lens 113 a and the optical axis ofthe third lens 113 c, or the distance between the optical axis of thesecond lens 113 b and the optical axis of the fourth lens 113 d, thesedistances being substantially equal. A is the distance from theprincipal point of a lens (principal point of the first lens 113 a,principal point of the second lens 113 b, principal point of the thirdlens 113 c, or principal point of the fourth lens 113 d) to the display203 (origin correction chart 206) in the imaging device. Note that thedistances from the principal points of the lenses to the display 203 aresubstantially equal. The origin correction coefficient generating unit252 removes parallax (Δx or Δy) from the respective deviations (s1 x, s1y), (s2 x, s2 y), (s3 x, s3 y) and (s4 x, s4 y) of the centers of thecrosses of the first imaging signal I1, the second imaging signal I2,the third imaging signal I3 and the fourth imaging signal 14 (centers ofsolid lines shown in FIG. 27A-27D) relative to the centers of the images(centers of broken lines shown in FIG. 27A-27D), and generates theorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y, as in the following equations (55), (56), (57), (58), (59), (60),(61) and (62). Note that in step S2330, image capture may be performed aplurality of times on the origin correction chart 206, and an imageobtained by averaging these images may be used as the imaging signal foruse in generating origin correction coefficients. In this case, theeffect of random noise and the like can be reduced by averaging. Also,the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y,g4 x, g4 y may be derived with decimal point precision, by deriving theparallax (Δx or Δy) or the deviations (s1 x, s1 y), (s2 x, s2 y), (s3 x,s3 y) and (s4 x, s4 y) with decimal point precision. Step S2620 isexecuted next.

Δx=f*Dx/A  (53)

Δy=f*Dy/A  (54)

g1x=s1x−Δx/2  (55)

g1y=s1y−Δy/2  (56)

g2x=s2x+Δx/2  (57)

g2y=s2y−Δy/2  (58)

g3x=s3x−Δx/2  (59)

g3y=s3y+Δy/2  (60)

g4x=s4x+Δx/2  (61)

g4x=s4y+Δy/2  (62)

In step S2620, the origin correction coefficient generating unit 252writes the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x,g3 y, g4 x, g4 y to the origin correction coefficient memory 138. Theseorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y are used in the origin correction of step S1400. Step S2710 isexecuted next.

In step S2710, the distortion correction coefficient generating unit 253generates the distortion correction coefficients p1 x(x, y), p1 y(x, y),p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y).Hereinafter, the method for generating the first distortion correctioncoefficient p1 x(x, y), p1 y(x, y) will be described. With the firstimaging signal I1(2, x, y) for use in generating distortion correctioncoefficients that result from capturing an image of the distortioncorrection chart 207, what should have been imaged as the lattice ofFIG. 29 with no distortion (broken lines) when the lens units are notdistorted is instead imaged as a distorted lattice (solid lines) due todistortion of the lens units. Firstly, the coordinates (ux1(i, j),uy1(i, j)) of the intersection of the distorted lattice (solid linesshown in FIG. 29) are derived by binarizing the first imaging signalI1(2, x, y) for use in distortion correction coefficient generation, anddetecting the cross pattern. The intersection (ux1(i, j), uy1(i, j)) ofthis distorted lattice (solid lines shown in FIG. 29) would be at theintersection (vx1(i, j), vy1(i, j)) of the undistorted lattice (brokenlines shown in FIG. 29) if the lens units were not distorted.

In the undistorted lattice (broken lines shown in FIG. 29), eachintersection (vx1(i, j), vy1(i, j)) of this lattice is expressed as inthe following equations (63) and (64), where (vx10, vy10) are thecoordinates of the upper-left intersection (vx1(0, 0), vy1(0, 0)), px isan interval of the lattice in the x direction, and py is an interval ofthe lattice in the y direction. vx10, vy10, px, py are determined fromthe focal distance of the lens, the distance between the imaging device201 and the display 203, the lattice size of the distortion correctionchart 207, or the like. Since the intersection (ux1(i, j), uy1(i, j)) ofthe distorted lattice (solid lines shown in FIG. 29) would be at theintersection (vx1(i, j), vy1(i, j)) of the undistorted lattice (brokenlines shown in FIG. 29) if there were no distortion, the firstdistortion correction coefficient p1 x(vx1(i, j), vy1(i, j)), p1y(vx1(i, j), vy1(i, j)) at the intersection (vx1(i, j), vy1(i, j)) is asshown in the following expressions (65) and (66). The first distortioncorrection coefficient p1 x(x, y), p1 y(x, y) for coordinates other thanthe intersection (vx1(i, j), vy1(i, j)) is generated by linearlyinterpolating the distortion correction coefficients (shown by equations(65), (66)) for intersections of the undistorted lattice (broken linesshown in FIG. 29) neighboring the coordinates (x, y), as in thefollowing equations (67) and (68). FIG. 31 shows the coordinatesreferenced when generating a distortion correction coefficient by linearinterpolation. Here, (vx1(i, j), vy1(i, j)) is the intersection of theundistorted lattice (broken lines shown in FIG. 29) on the upper-left ofthe coordinates (x, y) that are being derived. The distortion correctioncoefficient generating unit 253 then removes the effect of origindeviation and parallax using the origin correction coefficient g1 x, g1y and the parallaxes Δx (see equation (53)) and Δy (see equation (54)),as in the following equations (69) and (70).

$\begin{matrix}{{{vx}\; 1\left( {i,j} \right)} = {{{vx}\; 10} + {{px}*i}}} & (63) \\{{{vy}\; 1\left( {i,j} \right)} = {{{vy}\; 10} + {{py}*i}}} & (64) \\{{p\; 1{x\left( {{{vx}\; 1\left( {i,j} \right)},{{vy}\; 1\left( {i,j} \right)}} \right)}} = {{{ux}\; 1\left( {i,j} \right)} - {{vx}\; 1\left( {i,j} \right)}}} & (65) \\{{p\; 1{y\left( {{{vx}\; 1\left( {i,j} \right)},{{vy}\; 1\left( {i,j} \right)}} \right)}} = {{{uy}\; 1\left( {i,j} \right)} - {{vy}\; 1\left( {i,j} \right)}}} & (66) \\{{p\; 1{x\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 1\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 1\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 1{x\left( {{{vx}\; 1\left( {i,j} \right)},{{vy}\; 1\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 1\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 1\left( {{i + 1},j} \right)}} \right)*p\; 1{x\left( {{{vx}\; 1\left( {i,{j + 1}} \right)},{{vy}\; 1\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 1\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 1\left( {i,{j + 1}} \right)} - y} \right)*p\; 1{x\left( {{{vx}\; 1\left( {{i + 1},j} \right)},{{vy}\; 1\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 1\left( {i,j} \right)}} \right)*{y\left( {{vy}\; 1\left( {i,j} \right)} \right)}*p\; 1{x\left( {{{vx}\; 1\left( {{i + 1},{j + 1}} \right)},{{vy}\; 1\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (67) \\{{p\; 1{y\left( {x,y} \right)}} = \left\lbrack {{\left( {{{vx}\; 1\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 1\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 1{y\left( {{{vx}\; 1\left( {i,j} \right)},{{vy}\; 1\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 1\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 1\left( {{i + 1},j} \right)}} \right)*p\; 1{y\left( {{{vx}\; 1\left( {i,{j + 1}} \right)},{{vy}\; 1\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 1\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 1\left( {i,{j + 1}} \right)} - y} \right)*p\; 1{y\left( {{{vx}\; 1\left( {{i + 1},j} \right)},{{vy}\; 1\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 1\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 1\left( {i,j} \right)}} \right)*{\left( {p\; 1{y\left( {{{vx}\; 1\left( {{i + 1},{j + 1}} \right)},{{vy}\; 1\left( {{i + 1},{j + 1}} \right)}} \right)}} \right\rbrack/\left( {{px}*{py}} \right)}}} \right.} & (68) \\{{p\; 1{x\left( {x,y} \right)}} = {{p\; 1{x\left( {x,y} \right)}} - {g\; 1x} - {\Delta \; {x/2}}}} & (69) \\{{p\; 1{y\left( {x,y} \right)}} = {{p\; 1{y\left( {x,y} \right)}} - {g\; 1y} - {\Delta \; {y/2}}}} & (70)\end{matrix}$

The distortion correction coefficient generating unit 253 also similarlyderives the second distortion correction coefficient p2 x(x, y), p2 y(x,y), the third distortion correction coefficient p3 x(x, y), p3 y(x, y),and the fourth distortion correction coefficient p4 x(x, y), p4 y(x, y).That is, the coordinates (ux2(i, j), uy2(i, j)), (ux3(i, j), uy3(i, j))and (ux4(i, j), uy4(i, j)) of the intersections of the distorted latticeare derived based on the second imaging signal I2(2, x, y) for use indistortion correction coefficient generation, the third imaging signalI3(2, x, y) for use in distortion correction coefficient generation, andthe fourth imaging signal I4(2, x, y) for use in distortion correctioncoefficient generation. The intersections (vx2(i, j), vy2(i, j)),(vx3(i, j), vy3(i, j)) and (vx4(i, j), vy4(i, j)) of the undistortedlattice are as shown in the following equations (71), (72), (73), (74),(75) and (76).

The second distortion correction coefficient p2 x(vx2(i, j), vy2(i, j)),p2 y(vx2(i, j), vy2(i, j)) at (vx2(i, j), vy2(i, j)) is as shown in thefollowing equations (77) and (78), the third distortion correctioncoefficient p3 x(vx3(i, j), vy3(i, j)), p3 y(vx3(i, j), vy3(i, j)) at(vx3(i, j), vy3(i, j)) is as shown in the following equations (79) and(80), and the fourth distortion correction coefficient p4 x(vx4(i, j),vy4(i, j)), p4 y(vx4(i, j), vy4(i, j)) at (vx4(i, j), vy4(i, j)) is asshown in the following equations (81) and (82). Further, the distortioncorrection coefficient generating unit 253 generates the seconddistortion correction coefficient p2 x(x, y), p2 y(x, y) for coordinatesother than the intersection (vx2(i, j), vy2(i, j)) by linearlyinterpolating the distortion correction coefficients (shown by equations(77), (78)) for intersections of the undistorted lattice neighboring thecoordinates (x, y), as in the following equations (83) and (84). Thedistortion correction coefficient generating unit 253 also generates thethird distortion correction coefficient p3 x(x, y), p3 y(x, y) forcoordinates other than the intersection (vx3(i, j), vy3(i, j)) bylinearly interpolating the distortion correction coefficients (shown byequations (79), (80)) for intersections of the undistorted latticeneighboring the coordinates (x, y), as in the following equations (85)and (86). Further, the distortion correction coefficient generating unit253 generates the fourth distortion correction coefficient p4 x(x, y),p4 y(x, y) for coordinates other than the intersection (vx4(i, j),vy4(i, j)) by linearly interpolating the distortion correctioncoefficients (shown by equations (81), (82)) for intersections of theundistorted lattice neighboring the coordinates (x, y), as in thefollowing equations (87) and (88). The distortion correction coefficientgenerating unit 253 then removes the effect of origin deviation andparallax using the origin correction coefficient g2 x, g2 y, g3 x, g3 y,g4 x, g4 y and the parallaxes Δx (see equation (53)) and Δy (seeequation (54)), as in the following equations (89), (90), (91), (92),(93) and (94). Step S2720 is executed next.

$\begin{matrix}{{{vx}\; 2\left( {i,j} \right)} = {{{vx}\; 20} + {{px}*i}}} & (71) \\{{{vy}\; 2\left( {i,j} \right)} = {{{vy}\; 20} + {{py}*i}}} & (72) \\{{{vx}\; 3\left( {i,j} \right)} = {{{vx}\; 30} + {{px}*i}}} & (73) \\{{{vy}\; 3\left( {i,j} \right)} = {{{vy}\; 30} + {{py}*i}}} & (74) \\{{{vx}\; 4\left( {i,j} \right)} = {{{vx}\; 40} + {{px}*i}}} & (75) \\{{{vy}\; 4\left( {i,j} \right)} = {{{vy}\; 40} + {{py}*i}}} & (76) \\{{p\; 2{x\left( {{{vx}\; 2\left( {i,j} \right)},{{vy}\; 2\left( {i,j} \right)}} \right)}} = {{{ux}\; 2\left( {i,j} \right)} - {{vx}\; 2\left( {i,j} \right)}}} & (77) \\{{p\; 2{y\left( {{{vx}\; 2\left( {i,j} \right)},{{vy}\; 2\left( {i,j} \right)}} \right)}} = {{{uy}\; 2\left( {i,j} \right)} - {{vy}\; 2\left( {i,j} \right)}}} & (78) \\{{p\; 3{x\left( {{{vx}\; 3\left( {i,j} \right)},{{vy}\; 3\left( {i,j} \right)}} \right)}} = {{{ux}\; 3\left( {i,j} \right)} - {{vx}\; 3\left( {i,j} \right)}}} & (79) \\{{p\; 3{y\left( {{{vx}\; 3\left( {i,j} \right)},{{vy}\; 3\left( {i,j} \right)}} \right)}} = {{{uy}\; 3\left( {i,j} \right)} - {{vy}\; 3\left( {i,j} \right)}}} & (80) \\{{p\; 4{x\left( {{{vx}\; 4\left( {i,j} \right)},{{vy}\; 4\left( {i,j} \right)}} \right)}} = {{{ux}\; 4\left( {i,j} \right)} - {{vx}\; 4\left( {i,j} \right)}}} & (81) \\{{p\; 4{y\left( {{{vx}\; 4\left( {i,j} \right)},{{vy}\; 4\left( {i,j} \right)}} \right)}} = {{{uy}\; 4\left( {i,j} \right)} - {{vy}\; 4\left( {i,j} \right)}}} & (82) \\{{p\; 2{x\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 2\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 2\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 2{x\left( {{{vx}\; 2\left( {i,j} \right)},{{vy}\; 2\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 2\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 2\left( {{i + 1},j} \right)}} \right)*p\; 2{x\left( {{{vx}\; 2\left( {i,{j + 1}} \right)},{{vy}\; 2\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 2\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 2\left( {i,{j + 1}} \right)} - y} \right)*p\; 2{x\left( {{{vx}\; 2\left( {{i + 1},j} \right)},{{vy}\; 2\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 2\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 2\left( {i,j} \right)}} \right)*p\; 2{x\left( {{{vx}\; 2\left( {{i + 1},{j + 1}} \right)},{{vy}\; 2\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (83) \\{{p\; 2{y\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 2\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 2\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 2{y\left( {{vx}\; 2\left( {i,j} \right){vy}\; 2\left( {i,j} \right)} \right)}} + {\left( {{{vx}\; 2\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 2\left( {{i + 1},j} \right)}} \right)*p\; 2{y\left( {{{vx}\; 2\left( {i,{j + 1}} \right)},{{vy}\; 2\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 2\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 2\left( {i,{j + 1}} \right)} - y} \right)*p\; 2{y\left( {{{vx}\; 2\left( {{i + 1},j} \right)},{{vy}\; 2\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 2\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 2\left( {i,j} \right)}} \right)*p\; 2{y\left( {{{vx}\; 2\left( {{i + 1},{j + 1}} \right)},{{vy}\; 2\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (84) \\{{p\; 3{x\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 3\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 3\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 3{x\left( {{{vx}\; 3\left( {i,j} \right)},{{vy}\; 3\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 3\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 3\left( {{i + 1},j} \right)}} \right)*p\; 3{x\left( {{{vx}\; 3\left( {i,{j + 1}} \right)},{{vy}\; 3\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 3\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 3\left( {i,{j + 1}} \right)} - y} \right)*p\; 3{x\left( {{{vx}\; 3\left( {{i + 1},j} \right)},{{vy}\; 3\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 3\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 3\left( {i,j} \right)}} \right)*p\; 3{x\left( {{{vx}\; 3\left( {{i + 1},{j + 1}} \right)},{{vy}\; 3\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (85) \\{{p\; 3{y\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 3\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 3\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 3{y\left( {{{vx}\; 3\left( {i,j} \right)},{{vy}\; 3\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 3\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 3\left( {{i + 1},j} \right)}} \right)*p\; 3{y\left( {{{vx}\; 3\left( {i,{j + 1}} \right)},{{vy}\; 3\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 3\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 3\left( {i,{j + 1}} \right)} - y} \right)*p\; 3{y\left( {{{vx}\; 3\left( {{i + 1},j} \right)},{{vy}\; 3\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 3\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 3\left( {i,j} \right)}} \right)*p\; 3{y\left( {{{vx}\; 3\left( {{i + 1},{j + 1}} \right)},{{vy}\; 3\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (86) \\{{p\; 4{x\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 4\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 4\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 4{x\left( {{{vx}\; 4\left( {i,j} \right)},{{vy}\; 4\left( {i,j} \right)}} \right)}} + {\left( {{{vx}\; 4\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 4\left( {{i + 1},j} \right)}} \right)*p\; 4{x\left( {{{vx}\; 4\left( {i,{j + 1}} \right)},{{vy}\; 4\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 4\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 4\left( {i,{j + 1}} \right)} - y} \right)*p\; 4{x\left( {{{vx}\; 4\left( {{i + 1},j} \right)},{{vy}\; 4\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 4\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 4\left( {i,j} \right)}} \right)*p\; 4{x\left( {{{vx}\; 4\left( {{i + 1},{j + 1}} \right)},{{vy}\; 4\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (87) \\{{p\; 4{y\left( {x,y} \right)}} = {\left\lbrack {{\left( {{{vx}\; 4\left( {{i + 1},{j + 1}} \right)} - x} \right)*\left( {{{vy}\; 4\left( {{i + 1},{j + 1}} \right)} - y} \right)*p\; 4{y\left( {{vx}\; 4\left( {i,j} \right){vy}\; 4\left( {i,j} \right)} \right)}} + {\left( {{{vx}\; 4\left( {{i + 1},j} \right)} - x} \right)*\left( {y - {{vy}\; 4\left( {{i + 1},j} \right)}} \right)*p\; 4{y\left( {{{vx}\; 4\left( {i,{j + 1}} \right)},{{vy}\; 4\left( {i,{j + 1}} \right)}} \right)}} + {\left( {x - {{vx}\; 4\left( {i,{j + 1}} \right)}} \right)*\left( {{{vy}\; 4\left( {i,{j + 1}} \right)} - y} \right)*p\; 4{y\left( {{{vx}\; 4\left( {{i + 1},j} \right)},{{vy}\; 4\left( {{i + 1},j} \right)}} \right)}} + {\left( {x - {{vx}\; 4\left( {i,j} \right)}} \right)*\left( {y - {{vy}\; 4\left( {i,j} \right)}} \right)*p\; 4{y\left( {{{vx}\; 4\left( {{i + 1},{j + 1}} \right)},{{vy}\; 4\left( {{i + 1},{j + 1}} \right)}} \right)}}} \right\rbrack/\left( {{px}*{py}} \right)}} & (88) \\{{p\; 2{x\left( {x,y} \right)}} = {{p\; 2{x\left( {x,y} \right)}} - {g\; 2x} - {\Delta \; {x/2}}}} & (89) \\{{p\; 2{y\left( {x,y} \right)}} = {{p\; 2{y\left( {x,y} \right)}} - {g\; 2y} - {\Delta \; {y/2}}}} & (90) \\{{p\; 3{x\left( {x,y} \right)}} = {{p\; 3{x\left( {x,y} \right)}} - {g\; 3x} - {\Delta \; {x/2}}}} & (91) \\{{p\; 3{y\left( {x,y} \right)}} = {{p\; 3{y\left( {x,y} \right)}} - {g\; 3y} - {\Delta \; {y/2}}}} & (92) \\{{p\; 4{x\left( {x,y} \right)}} = {{p\; 4{x\left( {x,y} \right)}} - {g\; 4x} - {\Delta \; {x/2}}}} & (93) \\{{p\; 4{y\left( {x,y} \right)}} = {{p\; 4{y\left( {x,y} \right)}} - {g\; 4y} - {\Delta \; {x/2}}}} & (94)\end{matrix}$

In step S2720, the distortion correction coefficient generating unit 253writes the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139. These distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are used in thedistortion correction of step S1500. Step S2810 is executed next.

In step S2810, the system control unit 231 deletes the correctioncoefficient generation program. The correction coefficient generationprogram is only necessary when generating intensity correctioncoefficients, origin correction coefficients, and distortion correctioncoefficients, and is not required when capturing images of normalsubjects. Consequently, downloading the correction coefficientgeneration program at step S2120, and deleting the correctioncoefficient generation program at this step enables memory that can beutilized in normal usage to be increased. Step S2820 is executed next.

In step S2820, the imaging device cable 204 is disconnected from theimaging device 201. This is carried out by an industrial robot. Thisimaging device cable 204 is next connected to another imaging device andused in generating intensity correction coefficients, origin correctioncoefficients and distortion correction coefficients for that imagingdevice. Step S2900 is executed next.

In step S2900, the generation of intensity correction coefficients,origin correction coefficients, and distortion correction coefficientsis ended.

The imaging device 201 then operates similarly to Embodiment 1 when innormal usage.

As a result of being configured and operated as above, the imagingdevice of Embodiment 2 has the following effects.

The imaging device 201 of Embodiment 2, in the manufacturing process,captures an image of the intensity correction chart 205 in step S2230,generates the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y) and a4(x, y) in step S2510, and writes the intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y) and a4(x, y) to the intensitycorrection coefficient memory 137 in step S2520. Then, in normal usage,the imaging device 201 performs intensity correction based on theseintensity correction coefficients a1(x, y), a2(x, y), a3(x, y) and a4(x,y) in step S1300. Biasing of light intensity distribution is therebycompensated and the occurrence of false colors is suppressed, enablingfine images to be synthesized even if the variability in componentprecision or assembly is different for each imaging device 201.

Note that the imaging device 201 of Embodiment 2 may also have theeffect of suppressing the occurrence of false colors even if thesensitivity bias for the first imaging unit 123 a, the second imagingunit 123 b, the third imaging unit 123 c, and the fourth imaging unit123 d respectively differ.

The imaging device 201 of Embodiment 2, in the manufacturing process,captures an image of the origin correction chart 206 in step S2330,generates the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y in step S2610, and writes the origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y to theorigin correction coefficient memory 138 in step S2620. Then, in normalusage, the imaging device 201 performs origin correction based on theorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y in step 1400. Since origin deviation is thereby compensated,correct parallax derived, and image synthesis performed based on thiscorrect parallax, fine images can be synthesized even if the variabilityin component precision or assembly is different for each imaging device201.

The imaging device 201 of Embodiment 2, in the manufacturing process,captures an image of the distortion correction chart 207 in step S2430,generates the distortion correction coefficients p1 x(x, y), p1 y(x, y),p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y)in step S2710, and writes the distortion correction coefficients p1 x(x,y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x,y), p4 y(x, y) to the distortion correction coefficient memory 139 instep S2720. Then, in normal usage, the imaging device 201 performsdistortion correction based on these distortion correction coefficientsp1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y),p4 x(x, y), p4 y(x, y) in step S1500. Since the effect of distortion isthereby reduced, correct parallax derived, and image synthesis performedbased on this correct parallax, fine images can be synthesized even ifthe variability in component precision or assembly is different for eachimaging device 201.

Note that in the imaging device 201 of Embodiment 2, images werecaptured of the intensity correction chart 205, the origin correctionchart 206, and the distortion correction chart 207 drawn on the display203, although images may be captured of paper charts that have beenappropriately illuminated. Also, images may be captured of transmissivecharts such as glass or transparent resin that have been disposed infront of diffuse white illumination.

In Embodiment 2, the imaging device 201 generates the intensitycorrection coefficients a1, a2, a3 and a4 in step S2510, generates theorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y in step S2610, and generates the distortion correction coefficientsp1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y),p4 x(x, y), p4 y(x, y) in step S2710. However, the imaging device 201may instead transfer digitized imaging signals I1(0, x, y), I2(0, x, y),I3(0, x, y) and I4(0, x, y) for use in intensity correction coefficientgeneration, imaging signals I1(1, x, y), I2(1, x, y), I3(1, x, y) andI4(1, x, y) for use in origin correction coefficient generation, andimaging signals I1(2, x, y), I2(2, x, y), I3(2, x, y) and I4(2, x, y)for use in distortion correction coefficient generation to thecorrection coefficient generation controller 202, and the correctioncoefficient generation controller 202 may compute the intensitycorrection coefficients a1, a2, a3 and a4, the origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and thedistortion correction coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y),p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y), and transferthe resultant correction coefficients to the imaging device 201. In thiscase, the imaging device 201 may save the intensity correctioncoefficients a1, a2, a3 and a4 computed by the correction coefficientgeneration controller 202 to the intensity correction coefficient memory137, save the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y to the origin correction coefficient memory 138, andsave the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139.

In the imaging device 201 of Embodiment 2, the correction coefficientgeneration program is placed in the memory of the system control unit231, although the correction coefficient generation program may insteadbe stored in an external memory such as a flash memory like an SD card,and the system control unit 231 may execute this correction coefficientgeneration program by accessing the external memory via the input/outputunit 136. Similarly, the imaging signals captured at steps S2510, S2610,and S2710 may also be saved to an external memory.

The imaging device 201 of Embodiment 2 sets in step S2510 the reciprocalof imaging signals I1(0, x, y), I2(0, x, y), I3(0, x, y) and I4(0, x, y)for use in intensity correction as the intensity correction coefficientsa1, a2, a3 and a4, although a spatial LPF or a temporal LPF may be used.For example, a spatial LPF such as in equation (12) may be applied.

The imaging device 201 of Embodiment 2 respectively derives theintensity correction coefficients a1, a2, a3, a4, the origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y and thedistortion correction coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y),p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) in stepsS2510, S2610 and S2710 using imaging signals obtained by performingone-off image capture in steps S2230, S2330 and S2430, although theresult of averaging imaging signals obtained by performing image capturea plurality of times may be used. Using the average of a plurality ofcaptured imaging signals is equivalent to applying a temporal LPF. Sincethe effect of random noise and the like is reduced by applying this LPF,enabling precise intensity correction coefficients a1, a2, a3 and a4,origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y, and distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tobe generated, fineer images can be synthesized.

The imaging device 201 of Embodiment 2 generates the intensitycorrection coefficients a1(x, y), a2(x, y), a3(x, y) and a4(x, y) foreach pixel (x, y), although approximation may be used. For example, anapproximation obtained by multiplying intensity correction coefficientsexponentially expanded in the x and y directions as in the followingequation (95) or an approximation exponentially expanded as in thefollowing equation (96) may be determined as the first intensitycorrection coefficient, coefficients (a1 x 0, a1 x 1, a1 x 2, a1 x 3, a1xz, a1 y 0, a1 y 1, a1 y 2, a1 y 3, a1 yz or a100, a110, a101, a120,a111, a102, a130, a121, a112, a103, a1 xz, a1 yz) may be generated byleast squares or the like in step S2510, these coefficients may bewritten to the intensity correction coefficient memory 137 in stepS2520, and the first intensity correction value b1(x, y) for the pixel(x, y) may be generated using equations (95) or (96) in step S1320. Notethat the symbol “A” in equations (95) and (96) expresses exponentiation.The second intensity correction coefficient, the third intensitycorrection coefficient, and the fourth intensity correction coefficientmay be determined similarly to the first intensity correctioncoefficient, and in step S1320 the second intensity correction valueb2(x, y), the third intensity correction value b3(x, y) and the fourthintensity correction value b4(x, y) may be derived similarly toequations (95) and (96).

$\begin{matrix}{\left\lbrack {{a\; 1x\; 0} + {a\; 1x\; 1*\left( {x - {a\; 1{xz}}} \right)} + {a\; 1x\; 2*{\left( {x - {a\; 1{xz}}} \right)\bigwedge 2}} + {a\; 1x\; 3*{\left( {x - {a\; 1{xz}}} \right)\bigwedge 3}}} \right\rbrack*\left\lbrack {{a\; 1y\; 0} + {a\; 1y\; 1*\left( {y - {a\; 1{yz}}} \right)} + {a\; 1y\; 2*{\left( {y - {a\; 1{yz}}} \right)\bigwedge 2}} + {a\; 1y\; 3*{\left( {y - {a\; 1{yz}}} \right)\bigwedge 3}}} \right\rbrack} & (95) \\{{a\; 100} + {a\; 110*\left( {x - {a\; 1{xz}}} \right)} + {a\; 101*\left( {y - {a\; 1{yz}}} \right)} + {a\; 120*{\left( {x - {a\; 1{xz}}} \right)\bigwedge 2}} + {a\; 111*\left( {x - {a\; 1{xz}}} \right)*\left( {y - {a\; 1{yz}}} \right)} + {a\; 102*{\left( {y - {a\; 1{yz}}} \right)\bigwedge 2}} + {a\; 130*{\left( {x - {a\; 1{xz}}} \right)\bigwedge 3}} + {a\; 121*{\left( {x - {a\; 1{xz}}} \right)\bigwedge 2}*\left( {y - {a\; 1{yz}}} \right)} + {a\; 112*\left( {x - {a\; 1{xz}}} \right)*{\left( {y - \; {a\; 1{yz}}} \right)\bigwedge 2}} + {a\; 103*{\left( {y - {a\; 1{yz}}} \right)\bigwedge 3}}} & (96)\end{matrix}$

The imaging device 201 of Embodiment 2 captures an image of uniformwhite light of constant illuminance in step S2230, and generates one setof intensity correction coefficients a1(x, y), a2(x, y), a3(x, y) anda4(x, y), although a plurality of images may be captured of uniformwhite light of varying illuminance, and a plurality of sets of intensitycorrection coefficients may be generated. In this case, one of theplural sets of intensity correction coefficients may be selectedaccording to intensity in step S1300, and the selected set may be used.Also, a term that changes according to intensity may be added to theapproximation. In this case, nonlinear change due to intensity can alsobe compensated.

The imaging device 201 of Embodiment 2 captures an image of uniformwhite light of constant illuminance in step S2230, and generatesintensity correction coefficients a1(x, y), a2(x, y), a3(x, y) and a4(x,y), such that the imaging signals I1, I2, I3 and I4 are uniform,although the intensity correction coefficients may be generated toreduce intensity at the periphery of the image. That is, the result ofmultiplying a coefficient (1−ksh*[(x−L/2)̂2+(y−H/2)̂2]) that decreasestowards the periphery of the image by the ratio between a first imagingsignal I1(0, L/2, H/2) for use in intensity correction coefficientgeneration positioned at the center of the image and a first imagingsignal I1(0, x, y) for use in intensity correction coefficientgeneration is set as the first intensity correction coefficient a1(x,y), as in the following equation (97). Also, the result of multiplying acoefficient (1−ksh*[(x−L/2)̂2+(y−H/2)̂2]) that decreases towards theperiphery of the image by the ratio between a second imaging signalI2(0, L/2, H/2) for use in intensity correction coefficient generationpositioned at the center of the image and a second imaging signal I2(0p,x, y) for use in intensity correction coefficient generation is set asthe second intensity correction coefficient a2(x, y), as in thefollowing equation (98). Also, the result of multiplying a coefficient(1−ksh*[(x−L/2)̂2+(y−H/2)̂2]) that decreases towards the periphery of theimage by the ratio between a third imaging signal I3(0, L/2, H/2) foruse in intensity correction coefficient generation positioned at thecenter of the image and a third imaging signal I3(i, x, y) for use inintensity correction coefficient generation is set as the thirdintensity correction coefficient a3(x, y), as in the following equation(99). Further, the result of multiplying a coefficient(1−ksh*[(x−L/2)̂2+(y−H/2)̂2]) that decreases towards the periphery of theimage by the ratio between a fourth imaging signal I4(0, L/2, H/2) foruse in intensity correction coefficient generation positioned at thecenter of the image and a fourth imaging signal I4(i, x, y) for use inintensity correction coefficient generation is set as the fourthintensity correction coefficient a4(x, y), as in the following equation(100). Note that ksh shows a set value, (x−L/2)̂2 shows (x−L/2) squared,and (y−H/2)̂2 shows (y−H/2) squared. More natural images, in whichillumination around the periphery of the image has been reduced, canthereby be created.

$\begin{matrix}{{a\; 1\left( {x,y} \right)} = {I\; 1{\left( {0,{L/2},{H/2}} \right)/I}\; 1\left( {0,x,y} \right)*\left( {1 - {{ksh}*\left\lbrack \left( {x - {{L/2}\bigwedge 2} + {\left( {y - {H/2}} \right)\bigwedge 2}} \right\rbrack \right)}} \right.}} & (97) \\{{a\; 2\left( {x,y} \right)} = {I\; 2{\left( {0,{L/2},{H/2}} \right)/I}\; 2\left( {0,x,y} \right)*\left( {1 - {{ksh}*\left\lbrack {{\left( {x - {L/2}} \right)\bigwedge 2} + {\left( {y - {H/2}} \right)\bigwedge 2}} \right\rbrack}} \right)}} & (98) \\{{a\; 3\left( {x,y} \right)} = {I\; 3{\left( {0,{L/2},{H/2}} \right)/I}\; 3\left( {0,x,y} \right)*\left( {1 - {{ksh}*\left\lbrack {{\left( {x - {L/2}} \right)\bigwedge 2} + {\left( {y - {H/2}} \right)\bigwedge 2}} \right\rbrack}} \right)}} & (99) \\{{a\; 4\left( {x,y} \right)} = {I\; 4{\left( {0,{L/2},{H/2}} \right)/I}\; 4\left( {0,x,y} \right)*\left( {1 - {{ksh}*\left\lbrack {{\left( {x - {L/2}} \right)\bigwedge 2} + {\left( {y - {H/2}} \right)\bigwedge 2}} \right\rbrack}} \right)}} & (100)\end{matrix}$

As described above, the imaging device 201 of Embodiment 2 sets theratio between the first imaging signal I1(0, L/2, H/2) for use inintensity correction coefficient generation positioned at the center ofthe image and the first imaging signal I1(0, x, y) for use in intensitycorrection coefficient generation as the first intensity correctioncoefficient a1(x, y) in step S2510, as in equation (49), and sets theratio between the fourth imaging signal I4(0, L/2, H/2) for use inintensity correction coefficient generation positioned at the center ofthe image and the fourth imaging signal I4(i, x, y) for use in intensitycorrection coefficient generation as the fourth intensity correctioncoefficient a4(x, y), as in equation (52). The imaging device 201 thenuses these intensity correction coefficients, and calculates correctionvalues for each pixel (x, y) in step S1320. The imaging device 201 thensets the result of respectively multiplying the set values kab1 and kab4with the first intensity correction coefficient a1(x, y) and the fourthintensity correction coefficient a4(x, y) to the first intensitycorrection value b1(x, y) and the fourth intensity correction valueb4(x, y), as in equations (18) and (21). Further, the imaging device201, in step S1330, corrects the first imaging signal I1(x, y) and thefourth imaging signal I4(x, y), by respectively multiplying theseimaging signals by the first intensity correction value b1(x, y) and thefourth intensity correction value b4(x, y), as in equations (22) and(25).

Here, as a modification of Embodiment 2, the intensity levels of thefirst imaging signal I1 and the fourth imaging signal I4 may be madeequal. That is, the ratio between the first imaging signal I1(0, L/2,H/2) for use in intensity correction coefficient generation positionedat the center of the image and the fourth imaging signal I4(i, x, y) foruse in intensity correction coefficient generation may be set as thefourth intensity correction coefficient a4(x, y), as in the followingequation (101) rather than equation (52). Also, the result ofmultiplying the set value kab1 by the fourth intensity correctioncoefficient a4(x, y) may be set as the fourth intensity correction valueb4(x, y), as in the following equation (102) rather than equation (21).Also, the fourth intensity correction coefficient a4(x, y) may becorrected by being multiplied by the fourth imaging signal I4(x, y), asin equation (25). Since the intensity levels of the first imaging signalI1 and the fourth imaging signal I4 are thereby made equal, truerparallax is derived and image synthesis is performed based on the thistrue parallax, enabling fineer images to be synthesized even if there isvariability in the intensity levels of the first imaging signal I1 andthe fourth imaging signal I4 due to variability in the imaging elementor assembly.

a4(x,y)=I1(0,L/2,H/2)/I4(0,x,y)  (101)

b4(x,y)=kab1*a4(x,y)  (102)

Note that merely matching the levels of the first imaging element andthe fourth imaging element has the effect of equalizing the intensitylevels. That is, where the light intensity distributions are equal, andonly the average intensity levels differ, parallax precision improveseven when only the intensity levels are corrected. Specifically, thefirst intensity correction coefficient a1 and the fourth intensitycorrection coefficient a4 may be set as constants (not dependant on x ory), the set values kab1 and kab4 may be made the same, and the ratiobetween the first intensity correction coefficient a1 and the fourthintensity correction coefficient a4 may be set as the ratio between theaverage of the fourth imaging signals I4 (avg(I4)) and the average ofthe fourth imaging signals I1 (avg(I1)), as in the following equation(103). Since the intensity correction coefficients are thereby set asconstants, enabling the capacity of the intensity correction coefficientmemory 137 to be cut, cost reductions can be achieved.

a1/a4=avg(I4)/avg(I1)  (103)

The imaging device 201 of Embodiment 2 generates the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) for each pixel (x, y),although approximation may be used. For example, an approximationobtained by multiplying distortion correction coefficients exponentiallyexpanded in the x and y directions as in the following equation (104) oran approximation exponentially expanded as in the following equation(105) may be determined as the first distortion correction coefficient,coefficients (p1 x 0, p1 x 1, p1 x 2, p1 x 3, p1 xz, p1 y 0, p1 y 1, p1y 2, p1 y 3, p1 yz or p100, p110, p101, p120, p111, p102, p130, p121,p112, p103, p1 xz, p1 yz) may be generated by least squares or the likein step S2710, and these coefficients may be written to the distortioncorrection coefficient memory 139 in step S2720. Also, in step S1520,the first distortion correction coordinates (q1 x(x, y), q1 y(x, y)) maybe generated using equations (104) and (105) and the coordinates (x, y)of each pixel. Note that the symbol in equations (104) and (105)expresses exponentiation. The second distortion correction coefficient,the third distortion correction coefficient and the fourth distortioncorrection coefficient may be determined similarly to the firstdistortion correction coefficient, and in step S1520 the seconddistortion correction coordinates (q2 x(x, y), q2 y(x, y)), the thirddistortion correction coordinates (q3 x(x, y), q3 y(x, y)) and thefourth distortion correction coordinates (q4 x(x, y), q4 y(x, y)) may bederived using equations (104) and (105) and the coordinates (x, y) ofeach pixel, similarly to the first distortion correction coordinates (q1x(x, y), q1 y(x, y)).

$\begin{matrix}{\left\lbrack {{p\; 1x\; 0} + {p\; 1x\; 1*\left( {x - {p\; 1{xz}}} \right)} + {p\; 1x\; 2*{\left( {x - {p\; 1{xz}}} \right)\bigwedge 2}} + {p\; 1x\; 3*{\left( {x - {p\; 1{xz}}} \right)\bigwedge 3}}} \right\rbrack*\left\lbrack {{p\; 1y\; 0} + {p\; 1y\; 1*\left( {y - {p\; 1{yz}}} \right)} + {p\; 1y\; 2*{\left( {y - {p\; 1{yz}}} \right)\bigwedge 2}} + {p\; 1y\; 3*{\left( {y - {p\; 1{yz}}} \right)\bigwedge 3}}} \right\rbrack} & (104) \\{{p\; 100} + {p\; 110*\left( {x - {p\; 1{xz}}} \right)} + {p\; 101*\left( {y - {p\; 1{yz}}} \right)} + {p\; 120*{\left( {x - {p\; 1{xz}}} \right)\bigwedge 2}} + {p\; 111*\left( {x - {p\; 1{xz}}} \right)*\left( {y - {p\; 1{yz}}} \right)} + {p\; 102*{\left( {y - {p\; 1{yz}}} \right)\bigwedge 2}} + {p\; 130*{\left( {x - {p\; 1{xz}}} \right)\bigwedge 3}} + {p\; 121*{\left( {x - {p\; 1{xz}}} \right)\bigwedge 2}*\left( {y - {p\; 1{yz}}} \right)} + {p\; 112*\left( {x - {p\; 1{xz}}} \right)*{\left( {y - {p\; 1{yz}}} \right)\bigwedge 2}} + {p\; 103*{\left( {y - {p\; 1{yz}}} \right)\bigwedge 3}}} & (105)\end{matrix}$

In Embodiment 2, the imaging device 201 captures an image of the origincorrection chart 206 which has a cross drawn thereon, although theorigin correction chart is not limited to this. For example, an imagemay be captured of a chart with a dot drawn in the central portionthereof, and origin correction coefficients may be derived by derivingthe coordinates of the dot. Alternatively, an image may be captured of achart with a circle drawn thereon, and origin correction coefficientsmay be derived by deriving the center of the circle.

In Embodiment 2, the imaging device 201 captures an image of thedistortion correction chart 207 that has a lattice drawn thereon,although the distortion correction chart is not limited to this. FIGS.32A and 32B are external views of distortion correction charts used ingenerating distortion correction coefficients in a modification ofEmbodiment 2 of the present invention. For example, an image may becaptured of a chart with a circle drawn thereon as in FIG. 32A, anddistortion correction coefficients may be derived by deriving thedistortion of the circle. An image may also be captured of a checkeredchart as in FIG. 32B, and distortion correction coefficients may bederived by deriving the intersections of the extracted edges.

In the imaging device 201 of Embodiment 2, only the intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y), a4(x, y), the origincorrection coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y,and the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) aregenerated, although other subjects may be drawn, and other correctioncoefficients may be generated. For example, an image of the intensitycorrection chart 205 may be captured similar to Embodiment 2, andcorrection coefficients may be generated for black dots (places wherethe imaging signals are always approximately zero) and white dots(places where the imaging signals are always extremely large). Also,correction coefficients for γ (gamma) correction may be generated.Further, an image of the distortion correction chart 207 may becaptured, and correction coefficients may be generated for correctingdifferences in magnification between the imaging signals (first imagingsignal I1, second imaging signal I2, third imaging signal I3, fourthimaging signal) or differences in rotation angle. Images may be drawnthat enable a plurality of these to be measured simultaneously, and aplurality of correction coefficients may be generated. Also, signalsobtained by capturing these plurality of different subjects may besaved, and correction coefficients may be generated later.

In Embodiment 2, the imaging element 123 is constituted by the firstimaging element 123 a, the second imaging element 123 b, the thirdimaging element 123 c and the fourth imaging element 123 d, and theimaging signal input unit 133 is constituted by the first imaging signalinput unit 133 a, the second imaging signal input unit 133 b, the thirdimaging signal input unit 133 c, and the fourth imaging signal inputunit 133 d. However, the imaging element 123 may be constituted by asingle imaging element, and four images may be formed at differentpositions on a light receiving surface thereof by the first to fourthlens units 113 a to 113 d. Also, the imaging signal input unit 133 maybe constituted by a single imaging signal input unit that receives asinput signals from the single imaging element 123. In this case, thefirst imaging signal I1, the second imaging signal I2, the third imagingsignal I3, and the fourth imaging signal I4 should be set by settingfour areas in data placed in the memory of the system control unit 231,and extracting the data corresponding to each area. Also, whengenerating correction coefficients in the manufacturing process, thefirst imaging signal I1, the second imaging signal I2, the third imagingsignal I3, and the fourth imaging signal I4 may be generated using theabove area settings, and during actual operation of the imaging device201, the first imaging signal I1, the second imaging signal I2, thethird imaging signal I3, and the fourth imaging signal 14 may begenerated using the areas corrected with the origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y as the abovearea settings.

Note that although the above description illustrates the configurationand operations of a device that performs various corrections on imagingsignals obtained through image capture and corrects parallax beforesynthesizing images from the imaging signals, the imaging device of thepresent invention can also be applied as a measuring device fordetecting distance to the subject. That is, the imaging device of thepresent invention can also be implemented as a device that calculatesdistance based on parallax obtained as aforementioned, and outputs theobtained distance, with practical application as a surveying device,inter-vehicular distance detecting device or the like being conceivable.That is, equation (1), when solved for distance A, is as shown inequation (46). Accordingly, the distance to the subject from the blockB_(i) is as calculated in equation (47), and the distance to the subjectfrom a pixel (x, y) included in the block B_(i) is as shown in equation(48), and saved in the memory of the system control unit 231. Note thatthe units of measurement are changed appropriately when the calculationsare performed. If the distance information A(x, y) is then outputexternally via the input/output unit 136, an imaging device thatfunctions as a measuring device for detecting distance can be realized.

Embodiment 3

The imaging device 201 of the aforementioned Embodiment 2 generatesthree types of correction coefficients from images captured of threetypes of charts. That is, the intensity correction coefficients a1(x,y), a2(x, y), a3(x, y), a4(x, y) are generated from an image captured ofthe intensity correction chart 205, the origin correction coefficientsg1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y are generated from animage captured of the origin correction chart 206, and the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are generated from animage captured of the distortion correction chart 207. The imagingdevice according to Embodiment 3 of the present invention creates threetypes of correction coefficients from images captured of two types ofcharts. That is, the intensity correction coefficients a1(x, y), a2(x,y), a3(x, y), a4(x, y) and the origin correction coefficients g1 x, g1y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y are generated from an imagecaptured of an intensity/origin correction chart, and the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are generated from animage captured of the distortion correction chart 207. Thus, the imagingdevice of Embodiment 3 is able to shorten the time taken to generatecorrection coefficients in the manufacturing process.

The imaging device according to Embodiment 3 of the present inventionwill be described with reference to the drawings. FIG. 33 is across-sectional view showing the configuration of the imaging deviceaccording to Embodiment 3 of the present invention. In FIG. 33, animaging device 301 has a lens module unit 110 and a circuit unit 320.

The lens module unit 110 has a lens barrel 111, an upper cover glass112, a lens 113, a fixed actuator portion 114, and a movable actuatorportion 115. The circuit unit 320 has a substrate 121, a package 122, animaging element 123, a package cover glass 124, and a system LSI(hereinafter, SLSI) 325. The configurations and operations apart fromthe SLSI 325 are similar to Embodiment 1, with the same referencenumerals attached and redundant description omitted.

FIG. 34 is a block diagram of the imaging device according to Embodiment3 of the present invention. The SLSI 325 has a system control unit 331,an imaging element drive unit 132, an imaging signal input unit 133, anactuator manipulated variable output unit 134, an image processing unit135, an input/output unit 136, an intensity correction coefficientmemory 137, an origin correction coefficient memory 138, a distortioncorrection coefficient memory 139, an intensity correction coefficientgenerating unit 351, an origin correction coefficient generating unit352, and a distortion correction coefficient generating unit 253. Thecircuit unit 320 has an amplifier 126 in addition to the aboveconfiguration.

In an inspection process during the manufacturing process after assemblyof the imaging device 301, the intensity correction coefficientgenerating unit 351 generates intensity correction coefficients a1(x,y), a2(x, y), a3(x, y), a4(x, y), and writes the intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y), a4(x, y) to the intensitycorrection coefficient memory 137. The origin correction coefficientgenerating unit 352 also generates origin correction coefficients g1 x,g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and writes the origincorrection coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 yto the origin correction coefficient memory 138. Further, the distortioncorrection coefficient generating unit 253 generates distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y), and writes thedistortion correction coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y),p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) to thedistortion correction coefficient memory 139.

FIG. 35 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 3 of thepresent invention. 301 is the imaging device according to Embodiment 3of the present invention, 302 is a correction coefficient generationcontroller, 203 is a display, and 204 is a cable for connecting theimaging device. The correction coefficient generation controller 302,which is constituted by a computer, performs coordinated control of theimaging device 301 and the display 204, and causes the imaging device301 to generate the intensity correction coefficients a1(x, y), a2(x,y), a3(x, y), a4(x, y), the origin correction coefficients g1 x, g1 y,g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and the distortion correctioncoefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y),p3 y(x, y), p4 x(x, y), p4 y(x, y). The display 203, which isconstituted by a CRT display or the like, is controlled by thecorrection coefficient generation controller 202, operates incoordination with the imaging device 301, and draws images (charts) usedin generating the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y), a4(x, y), the origin correction coefficients g1 x, g1 y, g2 x,g2 y, g3 x, g3 y, g4 x, g4 y, and the distortion correction coefficientsp1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y),p4 x(x, y), p4 y(x, y). The imaging device cable 204, which isconstituted by a USB cable or the like, connects the imaging device 301and the correction coefficient generation controller 302, feeds power tothe imaging device 301, and is the medium through which communication isperformed between the imaging device 301 and the correction coefficientgeneration controller 302.

The operations of the imaging device according to Embodiment 3 of thepresent invention will be described next. FIG. 36 is an external view ofan intensity/origin correction chart used in generating intensitycorrection coefficients and origin correction coefficients according toEmbodiment 3 of the present invention. As shown in FIG. 36, theintensity/origin correction chart 305 is a uniform white subject with across drawn thereon. The imaging device 301 is disposed so as todirectly oppose the intensity/origin correction chart 305, and thecenters of the optical axes of the plurality of lens of the imagingdevice 301 are disposed so as to coincide with the center of the crossin the intensity/origin correction chart 305. The origin correctioncoefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y aregenerated with a similar method to Embodiment 2, using the cross portionof imaging signals. The intensity correction coefficients a1(x, y),a2(x, y), a3(x, y), a4(x, y) are generated with a similar method toEmbodiment 2 with regard to portions other than the cross portion of theimaging signals, and are derived by interpolating from portions otherthan the cross portion with regard to the cross portion of the imagingsignals.

The distortion correction coefficients p1 x(x, y), p1 y(x, y), p2 x(x,y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) aregenerated with a similar method to Embodiment 2 by capturing an image ofthe distortion correction chart 207.

FIG. 37 is a flowchart showing the method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 3 of the presentinvention.

In step S3000, the imaging device 301 is disposed such that the centersof the optical axes of the plurality of lens units of the imaging device301 coincide with the center of the drawing area on the display 203, andthe generation of intensity correction coefficients, origin correctioncoefficients, and distortion correction coefficients is started. StepS2110 is executed next.

In step S2110, the imaging device cable 204 is connected to the imagingdevice 301. Description of this step, which is similar to Embodiment 2,is omitted. Step S2120 is executed next.

In step S2120, a correction coefficient generation program is downloadedto the imaging device 301. Description of this step, which is similar toEmbodiment 2, is omitted. Step S3210 is executed next.

In step S3210, the correction coefficient generation controller 302causes the intensity/origin correction chart 305, which has a crossdisposed on a background of uniform white light, to be drawn on thedisplay 203 (see FIG. 36). Here, the intensity/origin correction chart305 is drawn such that the center of the drawing area on the display203, that is, the center of each optical axis of the plurality of lensunits of the imaging device 301 coincides with the intersection of thecross. Step S3220 is executed next.

In step S3220, the correction coefficient generation controller 302transmits a command to start image capture to the imaging device 301 viathe imaging device cable 204. Step S3230 is executed next.

In step S3230, the imaging device 301 performs image capture. Theimaging device 301 executes this step in response to the command in stepS3220. Description of this operation, which is similar to step S1200, isomitted. The imaging signals are saved to the memory of the systemcontrol unit 331 as a first imaging signal I1(0, x, y), a second imagingsignal I2(0, x, y), a third imaging signal I3(0, x, y), and a fourthimaging signal I4(0, x, y) for use in generating intensity and origincorrection coefficients. Note that an area for saving moving images orthe like in normal usage of the imaging device 301 is used as thismemory. Step S2410 is executed next.

In step S2410, the correction coefficient generation controller 202draws the distortion correction chart 207. Description of this step,which is similar to Embodiment 2, is omitted. Step S2420 is executednext.

In step S2420, the correction coefficient generation controller 202transmits a command to start image capture to the imaging device 201 viathe imaging device cable 204. This step is similar to Embodiment 2. StepS2430 is executed next.

In step S2430, the imaging device 301 performs image capture.Description of this step, which is similar to Embodiment 2, is omitted.Step S3510 is executed next.

In step S3510, the intensity correction coefficient generating unit 351generates the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y) and a4(x, y). The intensity correction coefficient generatingunit 351 sets the reciprocal of the first imaging signal I1(0, x, y) foruse in intensity and origin correction coefficient generation as thefirst intensity correction coefficient a1(x, y) in relation to pixels inportions of the intensity/origin correction chart 305 other than wherethe cross is drawn, as in the following equation (106). Similarly, theintensity correction coefficient generating unit 351 sets the reciprocalof the second imaging signal I2(0p, x, y) for use in intensity andorigin correction coefficient generation as the second intensitycorrection coefficient a2(x, y) as in the following equation (107), setsthe reciprocal of the third imaging signal I3(i, x, y) for use inintensity and origin correction coefficient generation as the thirdintensity correction coefficient a3(x, y) as in the following equation(108), and sets the reciprocal of the fourth imaging signal I4(i, x, y)for use in intensity and origin correction coefficient generation as thefourth intensity correction coefficient a4(x, y) as in the followingequation (109). Next, the intensity correction coefficient generatingunit 351 derives the correction coefficients for pixels in portions ofthe intensity/origin correction chart 305 where the cross is drawn,using the intensity correction coefficients of nearest neighboringpixels where the cross is not drawn. That is, the intensity correctioncoefficient generating unit 351, in relation to portions where the crossis drawn, sets the first intensity correction coefficient a1(x, y) ofthe pixel (x, y) to the first intensity correction coefficient a1(xn1(x,y), yn1(x, y)) of the nearest neighboring pixel (xn1(x, y), yn1(x, y))where the cross is not drawn, as in the following equation (110).Similarly, the intensity correction coefficient generating unit 351 setsthe second intensity correction coefficient a2(x, y) of the pixel (x, y)to the second intensity correction coefficient a2(xn2(x, y), yn2(x, y))of the nearest neighboring pixel (xn2(x, y), yn2(x, y)) where the crossis not drawn, as in the following equation (111). Also, the intensitycorrection coefficient generating unit 351 sets the third intensitycorrection coefficient a3(x, y) of the pixel (x, y) to the thirdintensity correction coefficient a3(xn3(x, y), yn3(x, y)) of the nearestneighboring pixel (xn3(x, y), yn3(x, y)) where the cross is not drawn,as in the following equation (112). Further, the intensity correctioncoefficient generating unit 351 sets the fourth intensity correctioncoefficient a4(x, y) of the pixel (x, y) to the fourth intensitycorrection coefficient a4(xn1(x, y), yn4(x, y)) of the nearestneighboring pixel (xn4(x, y), yn4(x, y)) where the cross is not drawn,as in the following equation (113). Note that image capture may beperformed a plurality of times in step S3230, and an image obtained byaveraging these images may be used. In this case, the effect of randomnoise and the like can be reduced by averaging. Also, approximation maybe used, by using the average of the coefficients of the two nearestvertical neighboring pixels, using the average of the coefficients ofthe two nearest lateral neighboring pixels, using the average of thecoefficients of the four nearest vertical and lateral neighboringpixels, or predicting the coefficients by extrapolation. Step S2520 isexecuted next.

a1(x,y)=1/I1(0,x,y)  (106)

a2(x,y)=1/I2(0,x,y)  (107)

a3(x,y)=1/I3(0,x,y)  (108)

a4(x,y)=1/I4(0,x,y)  (109)

a1(x,y)=a1(xn1(x,y),yn1(x,y))  (110)

a2(x,y)=a2(xn2(x,y),yn2(x,y))  (111)

a3(x,y)=a3(xn3(x,y),yn3(x,y))  (112)

a4(x,y)=a4(xn4(x,y),yn4(x,y))  (113)

In step S2520, the intensity correction coefficient generating unit 351writes the intensity correction coefficients a1(x, y), a2(x, y), a3(x,y) and a4(x, y) to the intensity correction coefficient memory 137.These intensity correction coefficients a1(x, y), a2(x, y), a3(x, y) anda4(x, y) are used in the intensity correction of S1300. This step issimilar to Embodiment 2. Step S3610 is executed next.

In step S3610, the origin correction coefficient generating unit 352generates the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 x. Description of this step, which is performed with asimilar method to Embodiment 2, is omitted. The first imaging signalI1(0, x, y) for use in intensity and origin correction coefficientgeneration, the second imaging signal I2(0, x, y) for use in intensityand origin correction coefficient generation, the third imaging signalI3(0, x, y) for use in intensity and origin correction coefficientgeneration, and the fourth imaging signal I4(0, x, y) for use inintensity and origin correction coefficient generation, however, areused as the imaging signals. Step S2620 is executed next.

In step S2620, the origin correction coefficient generating unit 352writes the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x,g3 y, g4 x, g4 y to the origin correction coefficient memory 138. Theseorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y are used in the origin correction of step S1400. This step issimilar to Embodiment 2. Step S2710 is executed next.

In step S2710, the distortion correction coefficient generating unit 253generates the distortion correction coefficients p1 x(x, y), p1 y(x, y),p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y).Description of this step, which is similar to Embodiment 2, is omitted.Step S2720 is executed next.

In step S2720, the distortion correction coefficient generating unit 253writes the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139. These distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are used in thedistortion correction of step S1500. This step is similar to Embodiment2. Step S2810 is executed next.

In step S2810, the system control unit 331 deletes the correctioncoefficient generation program. Description of this step, which issimilar to Embodiment 2, is omitted. Step S2820 is executed next.

In step S2820, the imaging device cable 204 is disconnected from theimaging device 201. Description of this step, which is similar toEmbodiment 2, is omitted. Step S3900 is executed next.

In step S3900, the generation of intensity correction coefficients,origin correction coefficients, and distortion correction coefficientsis ended.

As a result of being configured and operated as described above, theimaging device of Embodiment 3 obtains similar effects to Embodiment 2.Further, the imaging device of Embodiment 3 is able to suppress thenumber of times that image capture is performed in the manufacturingprocess and shorten the tact time of the manufacturing process, sincethe intensity correction coefficients a1(x, y), a2(x, y), a3(x, y),a4(x, y) and the origin correction coefficients g1 x, g1 y, g2 x, g2 y,g3 x, g3 y, g4 x, g4 y are generated using the same imaging signalobtained by capturing an image of a single intensity/origin correctionchart 305.

Note that although the above description illustrates the configurationand operations of a device that performs various corrections on imagingsignals obtained through image capture and corrects parallax beforesynthesizing images from the imaging signals, the imaging device of thepresent invention can also be applied as a measuring device fordetecting distance to the subject. That is, the imaging device of thepresent invention can also be implemented as a device that calculatesdistance based on parallax obtained as aforementioned, and outputs theobtained distance, with practical application as a surveying device,inter-vehicular distance detecting device or the like being conceivable.That is, equation (1), when solved for distance A, is as shown inequation (46). Accordingly, the distance to the subject from the blockB_(i) is as calculated in equation (47), and the distance to the subjectfrom a pixel (x, y) included in the block B_(i) is as shown in equation(48), and saved in the memory of the system control unit 331. Note thatthe units of measurement are changed appropriately when the calculationsare performed. If the distance information A(x, y) is then outputexternally via the input/output unit 136, an imaging device thatfunctions as a measuring device for detecting distance can be realized.

Embodiment 4

The imaging device of the aforementioned Embodiment 2 generates threetypes of correction coefficients from images captured of three types ofcharts. That is, the intensity correction coefficients a1(x, y), a2(x,y), a3(x, y) and a4(x, y) are generated from an image captured of theintensity correction chart 205, the origin correction coefficients g1 x,g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y are generated from an imagecaptured of the origin correction chart 206, and the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are generated from animage captured of the distortion correction chart 207. In contrast, theimaging device according to Embodiment 4 of the present inventioncreates three types of correction coefficients from an image captured ofone type of chart. That is, the intensity correction coefficients a1(x,y), a2(x, y), a3(x, y) and a4(x, y), the origin correction coefficientsg1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and the distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are generated from acaptured image of an intensity/origin/distortion correction chart. Thus,the imaging device of Embodiment 4 is able to shorten the time taken togenerate correction coefficients in the manufacturing process.

The imaging device according to Embodiment 4 of the present inventionwill be described with reference to the drawings. FIG. 38 is across-sectional view showing the configuration of the imaging deviceaccording to Embodiment 4 of the present invention. In FIG. 38, animaging device 401 has a lens module unit 110 and a circuit unit 420.

The lens module unit 110 has a lens barrel 111, an upper cover glass112, a lens 113, a fixed actuator portion 114, and a movable actuatorportion 115. The circuit unit 420 has a substrate 121, a package 122, animaging element 123, a package cover glass 124, and a system LSI(hereinafter, SLSI) 425. The configurations and operations apart fromthe SLSI 425 are similar to Embodiment 1, with the same referencenumerals attached and redundant description omitted.

FIG. 39 is a block diagram of the imaging device according to Embodiment4 of the present invention. The SLSI 425 has a system control unit 431,an imaging element drive unit 132, an imaging signal input unit 133, anactuator manipulated variable output unit 134, an image processing unit135, an input/output unit 136, an intensity correction coefficientmemory 137, an origin correction coefficient memory 138, a distortioncorrection coefficient memory 139, an intensity correction coefficientgenerating unit 451, an origin correction coefficient generating unit452, and a distortion correction coefficient generating unit 453. Thecircuit unit 420 has an amplifier 126 in addition to the aboveconfiguration.

In an inspection process during the manufacturing process after assemblyof the imaging device 401, the intensity correction coefficientgenerating unit 451 generates intensity correction coefficients a1(x,y), a2(x, y), a3(x, y), a4(x, y), and writes the intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y), a4(x, y) to the intensitycorrection coefficient memory 137, as a result of configuring andoperating the imaging device 401 as described hereinafter. Also, theorigin correction coefficient generating unit 452 generates origincorrection coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x, g4 y,and writes the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y to the origin correction coefficient memory 138.Further, the distortion correction coefficient generating unit 453generates distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y), andwrites the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139.

FIG. 40 is an external view of the imaging device and other devices whengenerating correction coefficients according to Embodiment 4 of thepresent invention. 401 is the imaging device according to Embodiment 4of the present invention, 402 is a correction coefficient generationcontroller, 203 is a display, and 204 is a cable for connecting theimaging device. The correction coefficient generation controller 402,which is constituted by a computer, performs coordinated control of theimaging device 401 and the display 204, and causes the imaging device401 to generate the intensity correction coefficients a1(x, y), a2(x,y), a3(x, y), a4(x, y), the origin correction coefficients g1 x, g1 y,g2 x, g2 y, g3 x, g3 y, g4 x, g4 y, and the distortion correctioncoefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y),p3 y(x, y), p4 x(x, y), p4 y(x, y). The display 203, which isconstituted by a CRT display or the like, is controlled by thecorrection coefficient generation controller 402, operates incoordination with the imaging device 401, and draws images (charts) usedin generating the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y), a4(x, y), the origin correction coefficients g1 x, g1 y, g2 x,g2 y, g3 x, g3 y, g4 x, g4 y, and the distortion correction coefficientsp1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y),p4 x(x, y), p4 y(x, y). The imaging device cable 204, which isconstituted by a USB cable or the like, connects the imaging device 401and the correction coefficient generation controller 402, feeds power tothe imaging device 401, and is the medium through which communication isperformed between the imaging device 401 and the correction coefficientgeneration controller 402.

The operations of the imaging device according to Embodiment 4 of thepresent invention will be described next. FIG. 41 is an external view ofan intensity/origin/distortion correction chart used in generatingintensity correction coefficients, origin correction coefficients anddistortion correction coefficients according to Embodiment 4 of thepresent invention. As shown in FIG. 41, the intensity/origin/distortioncorrection chart 405 is a uniform white subject with a lattice drawnthereon. The imaging device 401 is disposed so as to directly oppose theintensity/origin/distortion correction chart 405, and the centers of theoptical axes of the plurality of lens of the imaging device 401 aredisposed so as to coincide with the center of the lattice in theintensity/origin/distortion correction chart. The distortion correctioncoefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y),p3 y(x, y), p4 x(x, y), p4 y(x, y) are generated with a similar methodto Embodiment 2, using the lattice portion of the imaging signals. Theorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y are generated by a similar method to Embodiment 2, using the crossportion at the center of the lattice of the imaging signals (crossconstituted by 405 x & 405 y in FIG. 41). The intensity correctioncoefficients a1(x, y), a2(x, y), a3(x, y), a4(x, y) are generated with asimilar method to Embodiment 2 with regard to portions other than thelattice portion of the imaging signals, and are derived by interpolatingfrom portions other than the lattice portion with regard to the latticeportion of the imaging signals.

FIG. 42 is a flowchart showing the method of generating intensitycorrection coefficients, origin correction coefficients, and distortioncorrection coefficients according to Embodiment 4 of the presentinvention.

In step S4000, the imaging device 401 is disposed such that the centersof the optical axes of the plurality of lens units of the imaging device401 coincide with the center of the drawing area on the display, and thegeneration of intensity correction coefficients, origin correctioncoefficients, and distortion correction coefficients is started. StepS2110 is executed next.

In step S2110, the imaging device cable 204 is connected to the imagingdevice 401. Description of this step, which is similar to Embodiment 2,is omitted. Step S2120 is executed next.

In step S2120, a correction coefficient generation program is downloadedto the imaging device 401. Description of this step, which is similar toEmbodiment 2, is omitted. Step S4210 is executed next.

In step S4210, the correction coefficient generation controller 402causes the intensity/origin/distortion correction chart 405, which has alattice disposed on a background of uniform white light, to be drawn onthe display 203. Here, the intensity/origin/distortion correction chart405 is drawn such that the center of drawing area of the display 203,that is, the center of each optical axis of the plurality of lens unitsof the imaging device 401 coincides with the center of the lattice(intersection of 405 x & 405 y in FIG. 41). Step S4220 is executed next.

In step S4220, the correction coefficient generation controller 402transmits a command to start image capture to the imaging device 401 viathe imaging device cable 204. Step S4230 is executed next.

In step S4230, the imaging device 401 captures an image of theintensity/origin/distortion correction chart 405. The imaging device 401executes this step in response to the command in step S4220. Descriptionof this operation, which is similar to step S1200, is omitted. Theimaging signals are saved to the memory of the system control unit 431as a first imaging signal I1(0, x, y), a second imaging signal I2(0, x,y), a third imaging signal I3(0, x, y), and a fourth imaging signalI4(0, x, y) for use in intensity, origin and distortion correction. Notethat an area for saving moving images or the like in normal usage of theimaging device 401 is used as this memory. Step S4510 is executed next.

In step S4510, the intensity correction coefficient generating unit 451generates the intensity correction coefficients a1(x, y), a2(x, y),a3(x, y) and a4(x, y). The intensity correction coefficient generatingunit 451 sets the reciprocal of the first imaging signal I1(0, x, y) foruse in intensity and origin correction as the first intensity correctioncoefficient a1(x, y) in relation to pixels in portions other than wherethe cross is drawn, as in the following equation (114). Similarly, theintensity correction coefficient generating unit 451 sets the reciprocalof the second imaging signal I2(0p, x, y) for use in intensity andorigin correction as the second intensity correction coefficient a2(x,y) as in the following equation (115), sets the reciprocal of the thirdimaging signal I3(i, x, y) for use in intensity and origin correction asthe third intensity correction coefficient a3(x, y) as in the followingequation (116), and sets the reciprocal of the fourth imaging signalI4(i, x, y) for use in intensity and origin correction as the fourthintensity correction coefficient a4(x, y), as in the following equation(117). Next, the intensity correction coefficient generating unit 451uses the intensity correction coefficients of nearest neighboring pixelswhere the lattice is not drawn for pixels in portions of theintensity/origin/distortion correction chart 405 where the lattice isdrawn. That is, the intensity correction coefficient generating unit451, in relation to portions where the lattice is drawn, sets the firstintensity correction coefficient a1(x, y) of the pixel (x, y) to thefirst intensity correction coefficient a1(xn1(x, y), yn1(x, y)) of thenearest neighboring pixel (xn1(x, y), yn1(x, y)) where the lattice isnot drawn, as in the following equation (118). Similarly, the intensitycorrection coefficient generating unit 451 sets the second intensitycorrection coefficient a2(x, y) of the pixel (x, y) to the secondintensity correction coefficient a2(xn2(x, y), yn2(x, y)) of the nearestneighboring pixel (xn2(x, y), yn2(x, y)) where the lattice is not drawn,as in the following equation (119). The intensity correction coefficientgenerating unit 451 also sets the third intensity correction coefficienta3(x, y) of the pixel (x, y) to the third intensity correctioncoefficient a3(xn3(x, y), yn3(x, y)) of the nearest neighboring pixel(xn3(x, y), yn3(x, y)) where the lattice is not drawn, as in thefollowing equation (120), and sets the fourth intensity correctioncoefficient a4(x, y) of the pixel (x, y) to the fourth intensitycorrection coefficient a4(xn4(x, y), yn4(x, y)) of the nearestneighboring pixel (xn4(x, y), yn4(x, y)) where the lattice is not drawn,as in the following equation (121). Note that image capture may beperformed a plurality of times in step S4230, and an image obtained byaveraging these images may be used. In this case, the effect of randomnoise and the like can be reduced by averaging. Also, approximation maybe used, by using the average of the coefficients of the two nearestvertical neighboring pixels, using the average of the coefficients ofthe two nearest lateral neighboring pixels, using the average of thecoefficients of the four nearest vertical and lateral neighboringpixels, or predicting the coefficients by extrapolation. Step S2520 isexecuted next.

a1(x,y)=1/I1(0,x,y)  (114)

a2(x,y)=1/I2(0,x,y)  (115)

a3(x,y)=1/I3(0,x,y)  (116)

a4(x,y)=1/I4(0,x,y)  (117)

a1(x,y)=a1(xn1(x,y),yn1(x,y))  (118)

a2(x,y)=a2(xn2(x,y),yn2(x,y))  (119)

a3(x,y)=a3(xn3(x,y),yn3(x,y))  (120)

a4(x,y)=a4(xn4(x,y),yn4(x,y))  (121)

In step S2520, the intensity correction coefficient generating unit 451writes the intensity correction coefficients a1(x, y), a2(x, y), a3(x,y) and a4(x, y) to the intensity correction coefficient memory 137.These intensity correction coefficients a1(x, y), a2(x, y), a3(x, y) anda4(x, y) are used in the intensity correction of S1300. This step issimilar to Embodiment 2. Step S4610 is executed next.

In step S4610, the origin correction coefficient generating unit 452generates the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y. Description of this step, which is performed with asimilar method to Embodiment 2, is omitted. However, the origincorrection coefficient generating unit 452 uses the first imaging signalI1(0, x, y) for use in intensity, origin and distortion correctioncoefficient generation, the second imaging signal I2(0, x, y) for use inintensity, origin and distortion correction coefficient generation, thethird imaging signal I3(0, x, y) for use in intensity, origin anddistortion correction coefficient generation, and the fourth imagingsignal I4(0, x, y) for use in intensity, origin and distortioncorrection coefficient generation as the imaging signals. Also, theorigin correction coefficient generating unit 452 uses 405 x and 405 yin FIG. 41 as the cross. Step S2620 is executed next.

In step S2620, the origin correction coefficient generating unit 452writes the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x,g3 y, g4 x, g4 y to the origin correction coefficient memory 138. Theseorigin correction coefficients g1 x, g1 y, g2 x, g2 y, g3 x, g3 y, g4 x,g4 y are used in the origin correction of step S1400. This step issimilar to Embodiment 2. Step S4710 is executed next.

In step S4710, the distortion correction coefficient generating unit 453generates the distortion correction coefficients p1 x(x, y), p1 y(x, y),p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y).Description of this step, which is similar to Embodiment 2, is omitted.The distortion correction coefficient generating unit 453, however, usesthe first imaging signal I1(0, x, y) for use in intensity, origin anddistortion correction coefficient generation, the second imaging signalI2(0, x, y) for use in intensity, origin and distortion correctioncoefficient generation, the third imaging signal I3(0, x, y) for use inintensity, origin and distortion correction coefficient generation, andthe fourth imaging signal I4(0, x, y) for use in intensity, origin anddistortion correction coefficient generation as the imaging signals.Step S2720 is executed next.

In step S2720, the distortion correction coefficient generating unit 453writes the distortion correction coefficients p1 x(x, y), p1 y(x, y), p2x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) tothe distortion correction coefficient memory 139. These distortioncorrection coefficients p1 x(x, y), p1 y(x, y), p2 x(x, y), p2 y(x, y),p3 x(x, y), p3 y(x, y), p4 x(x, y), p4 y(x, y) are used in thedistortion correction of step S1500. This step is similar to Embodiment2. Step S2810 is executed next.

In step S2810, the system control unit 431 deletes the correctioncoefficient generation program. Description of this step, which issimilar to Embodiment 2, is omitted. Step S2820 is executed next.

In step S2820, the imaging device cable 204 is disconnected from theimaging device 401. Description of this step, which is similar toEmbodiment 2, is omitted. Step S4900 is executed next.

In step S4900, the generation of intensity correction coefficients,origin correction coefficients, and distortion correction coefficientsis ended.

As a result of being configured and operated as described above, theimaging device of Embodiment 4 obtains similar effects to Embodiment 2.

Further, the imaging device 401 of Embodiment 4 is able to suppress thenumber of times that image capture is performed in the manufacturingprocess and shorten the tact time of the manufacturing process, sincethe intensity correction coefficients a1(x, y), a2(x, y), a3(x, y),a4(x, y), the origin correction coefficients g1 x, g1 y, g2 x, g2 y, g3x, g3 y, g4 x, g4 y, and the distortion correction coefficients p1 x(x,y), p1 y(x, y), p2 x(x, y), p2 y(x, y), p3 x(x, y), p3 y(x, y), p4 x(x,y), p4 y(x, y) are generated using the same imaging signal obtained bycapturing an image of a single intensity/origin/distortion correctionchart 405.

Note that although the above description illustrates the configurationand operations of a device that performs various corrections on imagingsignals obtained through image capture and corrects parallax beforesynthesizing images from the imaging signals, the imaging device of thepresent invention can also be applied as a measuring device fordetecting distance to the subject. That is, the imaging device of thepresent invention can also be implemented as a device that calculatesdistance based on parallax obtained as aforementioned, and outputs theobtained distance, with practical application as a surveying device,inter-vehicular distance detecting device or the like being conceivable.That is, equation (1), when solved for distance A, is as shown inequation (46). Accordingly, the distance to the subject from the blockB_(i) is as calculated in equation (47), and the distance to the subjectfrom a pixel (x, y) included in the block B_(i) is as shown in equation(48), and saved in the memory of the system control unit 431. Note thatthe units of measurement are changed appropriately when the calculationsare performed. If the distance information A(x, y) is then outputexternally via the input/output unit 136, an imaging device thatfunctions as a measuring device for detecting distance can be realized.

Embodiment 5

The imaging device of Embodiment 1 has four lens units, each of whichdetects a single color. The imaging device of Embodiment 5 has two lensunits, each of which detects all of the colors. Also, the imaging deviceof Embodiment 5 outputs not only image information but also distanceinformation calculated from parallax.

The imaging device according to Embodiment 5 of the present inventionwill be described with reference to the drawings.

FIG. 43 is a cross-sectional view showing the configuration of animaging device 501 according to Embodiment 5 of the present invention.In FIG. 43, the imaging device 501 has a lens module unit 510 and acircuit unit 520.

The lens module unit 510 has a lens barrel 511, an upper cover glass512, a lens 513, a fixed actuator portion 514, and a movable actuatorportion 515. The circuit unit 520 has a substrate 521, a package 522, animaging element 523, a package cover glass 524, and a system LSI(hereinafter, SLSI) 525.

The lens barrel 511 is cylindrical and formed by injection-moldingresin, and the inner surface thereof is lusterless black in order toprevent diffused reflection of light. The upper cover glass 512 isdiscoid, formed from transparent resin, and anchored to the top surfaceof the lens barrel 511 using adhesive or the like, and the surfacethereof is provided with a protective film for preventing damage causedby abrasion or the like and an antireflective film for preventingreflection of incident light.

FIG. 44 is a top view of the lens 513 of the imaging device according toEmbodiment 5 of the present invention. The lens 513 is substantiallydiscoid and formed from glass or transparent resin, and has a first lensunit 513 a and a second lens unit 513 b disposed therein. The X-axis andthe Y-axis are set as shown in FIG. 44, along the directions in whichthe first and second lens units 513 a and 513 b are disposed. Lightincident on the first lens unit 513 a and the second lens unit 513 bfrom an upper curved portion is emitted from a bottom curved portion,and two images are formed on the imaging element 523.

The fixed actuator portion 514 is anchored to the inner surface of thelens barrel 511 by adhesive or the like. The movable actuator portion515 is anchored to the outer periphery of the lens 513 by adhesive orthe like. Description of the fixed actuator portion 514 and the movableactuator portion 515, whose detailed configuration is similar to thefixed actuator portion 114 and the movable actuator portion 115 ofEmbodiment 1, is omitted.

The substrate 521 is constituted by a resin substrate, and is anchoredby adhesive or the like, with the bottom surface of the lens barrel 511contacting the top thereof. The circuit unit 520 is thus anchored to thelens module unit 510 to constitute the imaging device 501.

The package 522 is formed from resin having a metal terminal, and isanchored inside the lens barrel 511 by soldering or the like the metalterminal unit to the top surface of the substrate 521. The imagingelement 523 is constituted by a first imaging element 523 a and a secondimaging element 523 b. The first imaging element 523 a and the secondimaging element 523 b are solid state imaging elements such as CCDsensors or CMOS sensors, and are disposed such that the centers of thelight receiving surfaces thereof are substantially aligned with thecenters of the optical axes of the first lens unit 513 a and the secondlens unit 513 b, and such that the light receiving surfaces of theimaging elements are substantially perpendicular to the optical axes ofthe corresponding lens units. The terminals of the first imaging element523 a and the second imaging element 523 b are connected with gold wires527 by wire bonding to the metal terminal on a bottom portion of thepackage 522 on the inside thereof, and electrically connected to theSLSI 525 via the substrate 521. Light emitted from the first lens unit513 a and the second lens unit 513 b forms images on the light receivingsurfaces of the first imaging element 523 a and the second imagingelement 523 b, and electrical information converted from opticalinformation by a photodiode is output to the SLSI 525.

FIG. 45 is a top view of the circuit unit 520 of the imaging deviceaccording to Embodiment 5 of the present invention. The package coverglass 524 is flat, formed using transparent resin, and anchored to thetop surface of the package 522 by adhesive or the like. A shadingportion 524 e is provided on the top surface of the package cover glass524 by vapor deposition or the like.

Consequently, object light incident from a top portion of the first lensunit 513 a is emitted from a bottom portion of the first lens unit 513a, passes through a cover glass 524 a, and forms an image on the lightreceiving portion of the first imaging element 523 a. Object lightincident from a top portion of the second lens unit 513 b is emittedfrom a bottom portion of the second lens unit 513 b, passes through acover glass 524 b, and forms an image on the light receiving portion ofthe second imaging element 523 b.

The SLSI 525 controls the energizing of the coil of the movable actuatorportion 515, drives the imaging element 523, receives as inputelectrical information from the imaging element 523, performs variousimage processing, communicates with a host CPU, and outputs imagesexternally as described later.

The relationship between subject distance and parallax will be describednext. Since the camera module according to Embodiment 5 of the presentinvention has two lens units (first lens unit 513 a, second lens unit513 b), the relative position of the two object images respectivelyformed by the two lens units changes according to subject distance, asdescribed in Embodiment 1 (see equation (1)).

The operations of the imaging device according to Embodiment 5 of thepresent invention will be described next. FIG. 46 is a block diagram ofthe imaging device according to Embodiment 5 of the present invention.The SLSI 525 has a system control unit 531, an imaging element driveunit 532, an imaging signal input unit 533, an actuator manipulatedvariable output unit 534, an image processing unit 535, an input/outputunit 536, an intensity correction coefficient memory 537, an origincorrection coefficient memory 538, and a distortion correctioncoefficient memory 539. The circuit unit 520 has an amplifier 526 inaddition to the above configuration. The amplifier 526 applies a voltagethat depends on the output from the actuator manipulated variable outputunit 534 to the coil of the movable actuator portion 515.

The system control unit 531, which is constituted by a CPU, a memory andthe like, controls the overall SLSI 525.

The imaging element drive unit 532, which is constituted by a logiccircuit and the like, generates a signal for driving the imaging element523, and applies a voltage that depends on this signal to the imagingelement 523.

The imaging signal input unit 533 is constituted by a first imagingsignal input unit 533 a and a second imaging signal input unit 533 b.The first imaging signal input unit 533 a and the second imaging signalinput unit 533 b are each configured with a CDS circuit, an AGC and anADC connected in series. The first imaging signal input unit 533 a andthe second imaging signal input unit 533 b are respectively connected tothe first imaging element 523 a and the second imaging element 523 b,and receive as input electrical signals from the imaging elements,remove static noise using the CDS circuit, adjust gains using the AGC,convert the analog signals to digital values using the ADC, and writethe digital values to the memory of the system control unit 531.

The actuator manipulated variable output unit 534, which is constitutedby a DAC, outputs a voltage signal that depends on the voltage to beapplied to the coil of the movable actuator portion 515.

The image processing unit 535, which is configured to include a logiccircuit or a DSP, or both, performs various image processing, usinginformation saved in the memory of the system control unit 531. Theimage processing unit 535 has an autofocus control unit 541, anintensity correcting unit 542, an origin correcting unit 543, adistortion correcting unit 544, and a distance calculating unit 545.

The input/output unit 536 communicates with the host CPU (not shown),and outputs image signals to the host CPU, an external memory (notshown) and an external display device such as an LCD (not shown).

The intensity correction coefficient memory 537, which is constituted bya nonvolatile memory such as a flash memory or a FeRAM, saves intensitycorrection coefficients for use by the intensity correcting unit 542.The origin correction coefficient memory 538, which is constituted by anonvolatile memory such as a flash memory or a FeRAM, saves origincorrection coefficients for use by the origin correcting unit 543. Thedistortion correction coefficient memory 539, which is constituted by anonvolatile memory such as a flash memory or a FeRAM, saves distortioncorrection coefficients for use by the distortion correcting unit 544.

FIG. 47 is a flowchart showing the operations of the imaging deviceaccording to Embodiment 5 of the present invention. The imaging device501 is operated by the system control unit 531 of the SLSI 525 as perthis flowchart.

In step S5000, operations are started. For example, the imaging device501 starts operations as the result of the host CPU (not shown)detecting that a shutter button (not shown) has been pressed, andinstructing the imaging device 501 to start operations via theinput/output unit 536. Step S5100 is executed next.

In step S5100, the autofocus control unit 541 executes autofocuscontrols. Description of this step, which is similar to Embodiment 1, isomitted. Step S5200 is executed next.

In step S5200, an image is input. The imaging element drive unit 532outputs signals for operating an electronic shutter and performingtransfer as needed, as a result of instructions from the system controlunit 531. The first imaging signal input unit 533 a and the secondimaging signal input unit 533 b, in sync with signals generated by theimaging element drive unit 532, respectively receive as input imagingsignals, which are analog signals of images output by the first imagingelement 523 a and the second imaging element 523 b, remove static noiseusing the CDS, automatically adjust input gains using the AGC, convertthe analog signals to digital values using the ADC, and write thedigital values to the memory of prescribed addresses in the systemcontrol unit 531 as a first imaging signal I1(x, y) and a second imagingsignal I2(x, y). As shown in FIG. 13, I1(x, y) indicates the firstimaging signal of the x-th horizontal and y-th vertical pixel. The totalnumber of pixels is H×L, where H is the number of pixels in the heightdirection and L is the number of pixels in the length direction of theinput image, with x changing from 0 to L−1, and y changing from 0 toH−1. The second imaging signal I2(x, y) similarly indicates the secondimaging signal of the x-th horizontal and y-th vertical pixel. The totalnumber of pixels is H×L, where H is the number of pixels in the heightdirection and L is the number of pixels in the length direction of theinput image, with x changing from 0 to L−1, and y changing from 0 toH−1. Step S5300 is executed next.

In step S5300, the intensity correcting unit 542 corrects the firstimaging signal I1 and the second imaging signal I2 using intensitycorrection coefficients saved in the intensity correction coefficientmemory 537. The results are then written to the memory of the systemcontrol unit 531. Description of this step, which is similar to stepS1300 in Embodiment 1, is omitted. Slight changes are, however,necessary, such as not using the third imaging signal I3 and the fourthimaging signal I4, or the third intensity correction coefficient a3(x,y) and the fourth intensity correction coefficient a4(x, y), which areused in Embodiment 1. Step S5400 is executed next.

In step S5400, the origin correcting unit 543 corrects the first imagingsignal I1 and the second imaging signal I2 using origin correctioncoefficients saved in the origin correction coefficient memory 538. Theresults are then written to the memory of the system control unit 531.Description of this step, which is similar to step S1400 in Embodiment1, is omitted. Slight changes are, however, necessary, such as not usingthe third imaging signal I3 and the fourth imaging signal I4, or thethird origin correction coefficient g3 x, g3 y, and the fourth intensitycorrection coefficient g4 x, g4 y, which are used in Embodiment 1. StepS5500 is executed next.

In step S5500, the distortion correcting unit 544 corrects the firstimaging signal I1 and the second imaging signal I2 using distortioncorrection coefficients saved in the distortion correction coefficientmemory 539. The results are then written to the memory of the systemcontrol unit 531. Description of this step, which is similar to stepS1500 in Embodiment 1, is omitted. Slight changes are, however,necessary, such as not using the third imaging signal I3 and the fourthimaging signal I4, or the third distortion correction coefficient p3x(x, y), p3 y(x, y), and the fourth distortion correction coefficient p4x(x, y), p4 y(x, y), which are used in Embodiment 1. Step S5600 isexecuted next.

In step S5600, the distance calculating unit 545 executes distancecalculation. FIG. 48 is a flowchart showing the distance calculationoperation according to Embodiment 5 of the present invention. Theflowchart of FIG. 48 shows the operations of step S5600 in detail.

Firstly, in step S5620, the distance calculating unit 545 performs blockdividing. Description of this step, which is similar to step S1620 inEmbodiment 1, is omitted. Step S5630 is executed next.

In step S5630, the distance calculating unit 545 calculates a parallaxvalue for each block. Firstly, a parallax evaluation value (R_(0(k)),R_(1(k)), . . . , R_(i(k)), . . . R_(MN-1(k)), k=0, 1, . . . , kmax) iscalculated for each block (B₀, B₁, . . . , B_(i), . . . , B_(MN-1)).FIG. 49 illustrates a calculation area for calculating parallaxevaluation values in the imaging device according to Embodiment 5 of thepresent invention. The area shown by B_(i) (also shown as I1) is thei-th block derived at step S5620 from the first imaging signal I1. Thearea shown by I2 is an area in which B_(i) has been moved by k in the xdirection. The total sum of absolute differences shown by the followingexpression (122) then is calculated as a parallax evaluation valueR_(i)(k) for all image signals I1(x, y) and I2(x−k, y) of the respectiveareas. Here, ΣΣ shows the total sum of all pixels in the block B_(i).

R _(i(k)) =ΣΣI1(x,y)−I2(x−k,y)|  (122)

This parallax evaluation value R_(i(k)) shows the level of correlationbetween the first image signal I1 of the i-th block B_(i) and the secondimage signal I2 in an area removed by k in the x direction. The smallerthe value, the greater the correlation (similarity). As shown in FIG.20, the parallax evaluation value R_(i(k)) changes depending on thevalue of k, and is minimized when k=Δi. This shows that the image signalof the block obtained by moving the i-th block B_(i) of the first imagesignal I1 by (Δi, 0) in the x and y directions, respectively, is mostclosely correlated to (most closely resembles) the second image signalI2. Consequently, we know that the parallax in the x and y directionsbetween the first imaging signal I1 and the second imaging signal I2 inrelation to i-th block B_(i) is (Δi, 0). Hereinafter, this Δi will becalled the parallax Δi of the i-th block B_(i). The parallax Δi of B_(i)is thus derived from i=0 to i=M×N−1. Step S5640 is executed next.

In step S5640, the distance calculating unit 545 performs distancecalculation. Equation (1), when solved for distance A, is as shown inthe following equation (123). Accordingly, the distance to the subjectfrom the block B_(i) is as calculated in the following equation (124),and the distance to the subject from a pixel (x, y) included in theblock B_(i) is as in the following equation (125). The derived distancesare saved in the memory of the system control unit 531. Note that theunits of measurement are changed appropriately when the calculations areperformed. Step S5650 is executed next.

A=f*D/Δ  (123)

A _(i) =f*D/Δ _(i)  (124)

A(x,y)=A_(i) ((x, y) included in B_(i))  (125)

In step S5650, distance calculation is ended and processing returns tothe main routine. Accordingly, step S5700 of FIG. 47 is executed next.

In step S5700, the input/output unit 536 outputs the result. Theinput/output unit 536 outputs I1(x, y), A(x, y), which is data in thememory of the system control unit 531, to the host CPU (not shown) or anexternal display device (not shown). Step S5800 is executed next.

In step S5800, operations are ended.

As a result of being configured and operated as above, the imagingdevice 501 has the following effects.

The imaging device 501 according to Embodiment 5, in step S5300,generates the intensity correction values b1(x, y) and b2(x, y), whosedegree of correction changes depending on the position (x, y) of theimaging area, based on the intensity correction coefficients a1 and a2,corrects the imaging signals I1(x, y), I2(x, y), and compensates biasingof light intensity distribution. The imaging device 501 also divides thefirst imaging signal I1 into a plurality of blocks in step S5620, andderives a parallax for each block based on the corrected imaging signalsI1(x, y) and I2(x, y) in step S5630. The imaging device 501 calculatesdistance for each block based on the parallax in step S5640. Since theimaging device 501 thus compensates biasing of light intensitydistribution, derives correct parallax and performs distance calculationbased on this correct parallax, correct distance can be generated.

The imaging device 501 according to Embodiment 5 saves the origincorrection coefficients g1 x, g2 x, g1 y, g2 y to the origin correctioncoefficient memory 138, and corrects the origins of the imaging signalsI1(x, y) and I2(x, y) based on the origin correction coefficients g1 x,g2 x, g1 y, g2 y in step S5400.

The imaging device 501 derives a parallax for each block based on thecorrected imaging signals I1(x, y) and I2(x, y) in step S5630. Further,the imaging device 501 calculates distance for each block based on theseparallaxes in step S5640. Since correct parallax thus is derived anddistances calculated based on this correct parallax, the imaging device501 can generate correct distances.

The imaging device 501 according to Embodiment 5 saves the distortioncorrection coefficients p1 x(x, y), p2 x(x, y), p1 y(x, y), p2 y(x, y),and calculates the distortion correction coordinates q1 x(x, y), q2 x(x,y), q1 y(x, y), q2 y(x, y), based on the distortion correctioncoefficients p1 x(x, y), p2 x(x, y), p1 y(x, y), p2 y(x, y) in stepS5520. Further, the imaging device 501 corrects the imaging signal I1(x,y), 12(x, y) at the distortion correction coordinates q1 x(x, y), q2x(x, y), q1 y(x, y), q2 y(x, y) in step S5530, so as to reduce theeffect of distortion of the plurality of lens units, and derives aparallax for each block based on the corrected imaging signals I1(x, y)and I2(x, y) in step S5630. Also, the imaging device 501 calculatesdistance based on these parallaxes in step S5640. Since correct parallaxis thus derived and distances calculated based on this correct parallax,the imaging device 501 can generate correct distances.

Note that although the imaging device of Embodiment 5 uses a monochromeimaging element, a Bayer array of imaging elements may be used. In thiscase, slight changes are necessary, including, for example, calculatingluminance from a color image, performing intensity correction, origincorrection and distortion correction on this luminance, calculatingparallax, and calculating distance.

Note that in embodiments 1 to 5, a block is divided into a rectangularshape, although the present invention is not limited to this. Forexample, the edge may be detected, and the imaging signal may be dividedinto non-rectangular blocks based on the edge. The edge may also bedivided into a plurality of segments, and the parallax of these segmentsmay be derived, rather than deriving parallax for the area of eachblock. Further, blocks may be divided or joined based on the evaluationof parallax derived in a given block.

In the embodiments 1 to 5, focal point control may be omitted, and anactuator not included in the configuration. Where the lens used has avery long focal depth, the actuator does not need to be operated, sinceample error tolerance is provided in the distance between the lens andthe imaging element.

INDUSTRIAL APPLICABILITY

The imaging device of the present invention is useful in mobiletelephones with camera function, digital still cameras, surveillancecameras, and in-vehicle cameras, or in measuring devices for detectingdistance or the like, because of the possibilities for size and profilereductions.

1. An imaging device comprising: a plurality of lens units eachincluding at least one lens; a plurality of imaging areas correspondingone-to-one with the plurality of lens units, and each having a lightreceiving surface substantially perpendicular to an optical axisdirection of the corresponding lens unit; an imaging signal input unitthat receives as input a plurality of imaging signals each output from adifferent one of the plurality of imaging areas; an intensity correctioncoefficient saving unit that saves an intensity correction coefficient,which is information concerning intensity unevenness in the imagingareas; an intensity correcting unit that corrects an intensity of eachof the plurality of imaging signals using the intensity correctioncoefficient, so as to reduce an effect of intensity unevenness in theimaging areas; and a parallax calculating unit that derives a parallaxbetween images formed by the plurality of lens units, based on theimaging signals whose intensity has been corrected by the intensitycorrecting unit.
 2. The imaging device according to claim 1, furthercomprising: an optical element on a light path of light incident on atleast two of the plurality of imaging areas that has transmissioncharacteristics substantially centered on a first wavelength; and anoptical element on a light path of light incident on the remainingimaging areas that has transmission characteristics substantiallycentered on a different wavelength from the first wavelength.
 3. Theimaging device according to claim 2, wherein the intensity correctingunit corrects the intensity of at least the imaging signalscorresponding to the imaging areas, of the plurality of the imagingareas, that receive light passing through the optical elements havingtransmission characteristics substantially centered on the firstwavelength.
 4. The imaging device according to claim 2, wherein thefirst wavelength is perceived as substantially green by human vision. 5.The imaging device according to claim 1, further comprising: a parallaxcalculating unit that derives a parallax between images formed by theplurality of lens units, based on the imaging signals whose intensityhas been corrected by the intensity correcting unit; and a parallaxcorrecting unit that corrects the plurality of imaging signals andperforms image synthesis based on the parallax.
 6. The imaging deviceaccording to claim 1, further comprising: a parallax calculating unitthat derives a parallax between images formed by the plurality of lensunits, based on the imaging signals whose intensity has been correctedby the intensity correcting unit; and a distance calculating unit thatderives a distance to a subject based on the parallax.
 7. The imagingdevice according to claim 5, further comprising: a block dividing unitthat divides at least one of the plurality of imaging signals into aplurality of blocks, wherein the parallax calculating unit calculatesthe parallax between images formed by the plurality of lens units foreach block.
 8. The imaging device according to claim 1, furthercomprising: an origin correction coefficient saving unit that saves anorigin correction coefficient, which is information concerningcorrespondence between an origin of the optical axes of the plurality oflens units and an origin of the imaging signals; and an origincorrecting unit that corrects an origin of each of the plurality ofimaging signals based on the origin correction coefficient.
 9. Theimaging device according to claim 1, further comprising: a distortioncorrection coefficient saving unit that saves a distortion correctioncoefficient, which is information concerning distortion of the lensunits; and a distortion correcting unit that corrects each of theplurality of imaging signals based on the distortion correctioncoefficient, so as to reduce an effect of distortion of the plurality oflens units.
 10. The imaging device according to claim 1, wherein theintensity correcting unit corrects the plurality of imaging signals suchthat intensity levels are equal.
 11. A manufacturing method for animaging device that has a plurality of lens units each including atleast one lens, a plurality of imaging areas corresponding one-to-onewith the plurality of lens units, and each having a light receivingsurface substantially perpendicular to the optical axis direction of thecorresponding lens unit, an imaging signal input unit that receives asinput a plurality of imaging signals each output from a different one ofthe imaging areas, an intensity correction coefficient saving unit thatsaves an intensity correction coefficient, which is informationconcerning intensity unevenness in the imaging areas, an intensitycorrecting unit that corrects an intensity of the imaging signals usingthe intensity correction coefficient, so as to reduce the effect ofintensity unevenness in the imaging areas, and a parallax calculatingunit that derives a parallax between images formed by the plurality oflens units, based on the imaging signals whose intensity has beencorrected by the intensity correcting unit, the manufacturing methodcomprising: a first image capturing step of using the imaging device tocapture an image of a substantially white object; an intensitycorrection coefficient calculating step of calculating the intensitycorrection coefficient based on an imaging signal obtained in the firstimage capturing step; and a step of saving the intensity correctioncoefficient calculated in the intensity correction coefficientcalculating step to the intensity correction coefficient saving unit.12. The manufacturing method according to claim 11, wherein the imagingdevice further includes an origin correction coefficient saving unitthat saves an origin correction coefficient, which is informationconcerning correspondence between an origin of the optical axes of theplurality of lens units and an origin of the imaging signals, and anorigin correcting unit that corrects an origin of the imaging signalsbased on the origin correction coefficient, and the manufacturing methodfurther comprises: a second image capturing step of using the imagingdevice to capture an image of an object having a pattern that includes across in a central portion thereof; an origin correction coefficientcalculating step of calculating the origin correction coefficient basedon an imaging signal obtained in the second image capturing step; and astep of saving the origin correction coefficient calculated in theorigin correction coefficient calculating step to the origin correctioncoefficient saving unit.
 13. The manufacturing method according to claim11, wherein the imaging device further includes a distortion correctioncoefficient saving unit that saves a distortion correction coefficient,which is information concerning distortion of the lens units, and adistortion correcting unit that corrects the imaging signals based onthe distortion correction coefficient, so as to reduce an effect ofdistortion of the plurality of lens units, and the manufacturing methodfurther comprises: a third image capturing step of using the imagingdevice to capture an image of an object having a lattice pattern; adistortion correction coefficient calculating step of calculating thedistortion correction coefficient based on an imaging signal obtained inthe third image capturing step; and a step of saving the distortioncorrection coefficient calculated in the distortion correctioncoefficient calculating step to the distortion correction coefficientsaving unit.
 14. The manufacturing method according to claim 11, whereinthe imaging device further includes an origin correction coefficientsaving unit that saves an origin correction coefficient, which isinformation concerning correspondence between an origin of the opticalaxes of the plurality of lens units and an origin of the imagingsignals, and an origin correcting unit that corrects an origin of theimaging signals based on the origin correction coefficient, an objecthaving a substantially white background and a pattern that includes across in a central portion thereof is used as the object in the firstimage capturing step, and the manufacturing method further comprises: anorigin correction coefficient calculating step of calculating the origincorrection coefficient based on the imaging signal obtained in the firstimage capturing step; and a step of saving the origin correctioncoefficient calculated in the origin correction coefficient calculatingstep to the origin correction coefficient saving unit.
 15. Themanufacturing method according to claim 11, wherein the imaging devicefurther includes a distortion correction coefficient saving unit thatsaves a distortion correction coefficient, which is informationconcerning distortion of the lens units, and a distortion correctingunit that corrects the imaging signals based on the distortioncorrection coefficient, so as to reduce an effect of distortion of theplurality of lens units, an object having a substantially whitebackground and a lattice pattern is used as the object in the firstimage capturing step, and the manufacturing method further comprises: adistortion correction coefficient calculating step of calculating thedistortion correction coefficient based on the imaging signal obtainedin the first image capturing step; and a step of saving the distortioncorrection coefficient calculated in the distortion correctioncoefficient calculating step to the distortion correction coefficientsaving unit.
 16. The manufacturing method according to claim 11, whereinthe imaging device further includes an origin correction coefficientsaving unit that saves an origin correction coefficient, which isinformation concerning correspondence between an origin of the opticalaxes of the plurality of lens units and an origin of the imagingsignals, an origin correcting unit that corrects an origin of theimaging signals based on the origin correction coefficient, a distortioncorrection coefficient saving unit that saves a distortion correctioncoefficient, which is information concerning distortion of the lensunits, and a distortion correcting unit that corrects the imagingsignals based on the distortion correction coefficient, so as to reducean effect of distortion of the plurality of lens units, an object havinga substantially white background and a lattice pattern is used as theobject in the first image capturing step, and the manufacturing methodfurther comprises: an origin correction coefficient calculating step ofcalculating the origin correction coefficient based on the imagingsignal obtained in the first image capturing step; a distortioncorrection coefficient calculating step of calculating the distortioncorrection coefficient based on the imaging signal obtained in the firstimage capturing step; and a step of saving the origin correctioncoefficient calculated in the origin correction coefficient calculatingstep to the origin correction coefficient saving unit, and saving thedistortion correction coefficient calculated in the distortioncorrection coefficient calculating step to the distortion correctioncoefficient saving unit.